& 


f 


Digitized  by  the  Internet  Archive 
in  2019  with  funding  from 
Princeton  Theological  Seminary  Library 


/ 


https://archive.org/details/tentativestandarOOmean 


Vol.  XXX 
No. 


Whole  No.  134 
1921 

!  JUt  27 


PSYCHOLOGICAL  REVIEW  PUBLICATIONS 


Psychological  Monographs 


EDITED  BY 


JAMES  ROWLAND  ANGELL,  522  Fifth  Avenue,  New  York. 

HOWARD  C.  WARREN,  Princeton  University  ( Review ) 

JOHN  B.  WATSON,  Johns  Hopkins  University  (/.  of  Exp.  Psychol .) 
SHEPHERD  I.  FRANZ,  Govt.  Hosp.  for  Insane  ( Bulletin )  and 


MADISON  BENTLEY,  University  of  Illinois  (Index) 


A  Tentative  Standardization  of  a 
Hard  Opposites  Test 


BY 


✓ 


MARIE  HACKL  MEANS,  Ph.D. 


PSYCHOLOGICAL  REVIEW  COMPANY 

PRINCETON,  N.  J. 
axd  LANCASTER,  PA. 

Agents:  G.  E.  STECHERT  &  CO.,  London  (2  Star  Yard,  Carey  St.,  W.  C.) 


Paris  (16,  rue  de  Conde) 


Grateful  acknowledgment  is  due  Dr.  Edward  K.  Strong  who 
suggested  the  problem  and  Dr.  Joseph  Peterson  under  whose 
guidance  and  help  the  work  was  completed.  The  writer  also 
wishes  to  thank  those  students  of  Peabody  College  who  acted  as 
subjects,  as  well  as  the  following  men  and  women  who  so  kindly 
gave  of  their  time  to  make  the  completion  of  this  research  pos¬ 
sible  :  Dr.  Harvey  Carr,  Miss  Cora  Bratton,  Dr.  George  Arps, 
Miss  Mary  Small,  Dr.  Walter  Hunter,  Dr.  Florence  Richard¬ 
son,  Dr.  Benjamin  Simpson,  Dr.  Norman  Triplett,  Dr.  Franklin 
Smith,  Dr.  Homer  Bean,  Dr.  E.  A.  Gamble,  Mr.  E.  A.  Kirk¬ 
patrick,  Dr.  C.  A.  Ruckmick,  Dr.  M.  R.  Trabue,  Dr.  George  W. 
Camp,  Dr.  W.  B.  Pillsbury. 

It  would  be  most  ungrateful  for  the  writer  not  to  acknowledge 
the  help  and  encouragement  received  from  her  mother. 


I 


CONTENTS 

Section  Page 

I  The  Problem  and  its  Importance .  i 

II  Results  of  Previous  Investigators .  2 

III  Previous  Efforts  toward  Standardization .  16 

IV  The  Selection  of  the  Stimuli  for  the  Present  Investi¬ 

gation  .  19 

V  The  Choice  of  the  Acceptable  Responses .  24 

VI  The  Computation  of  the  Relative  Difficulty  of  the 

Stimuli  .  33 

VII  Conversion  into  a  Group  Test .  48 

VIII  Establishment  of  Norms .  51 

IX  Interpretation  of  the  Results .  57 

X  Suggestions  and  Directions  for  the  Use  of  the  Oppo¬ 
sites  Test .  59 

XI  Summary  and  Conclusions .  62 

XII  Appendix  .  64 

Bibliography  . 65 


A  TENTATIVE  STANDARDIZATION  OF  A 
HARD  OPPOSITES  TEST 


I. 

The  Problem  and  its  Importance 

There  have  been  many  attempts  to  apply  mental  tests  to  college 
students,  but  the  results  have  not  been  on  the  whole  satisfactory. 
The  very  homogeneity  of  the  college  group,  brought  about  by  the 
natural  process  of  the  elimination  of  the  unfit,  should  have 
warned  us  not  to  expect  results  comparable  to  those  obtained 
from  children  and  unselected  adults.  The  minute  individual 
differences  of  degrees  of  accuracy  or  rate  of  performance  can  be 
detected  only  by  the  use  of  a  most  delicate  scale. 

Therefore  to  raise  our  correlations  of  mental  tests  and  esti¬ 
mated  intelligence  it  is  necessary  that  greater  attention  should  be 
paid  to  the  selection  of  the  tests  themselves.  Where  the  higher 
functions  are  involved  it  is  better  to  avoid  tests  of  co-ordination 
and  sense  discrimination  as  well  as  those  tests,  success  in  which 
is  largely  dependent  upon  speed  of  performance  rather  than  ac¬ 
curacy.  Unfortunately  the  humbler  task  of  standardization  for 
convenient  use  has  been  neglected  and  until  investigators  have 
at  their  disposal  a  number  of  such  tests,  our  attitude  toward  the 
validity  of  mental  testing  of  college  students  should  be  highly 
tentative. 

This  present  investigation  was  attempted  in  the  hope  of  con¬ 
tributing  a  small  amount  toward  the  preparation  of  some  ade¬ 
quately  standardized  tests.  To  evolve  out  of  a  test  in  common 
use  a  vastly  more  difficult  test,  to  determine  the  relative  dif¬ 
ficulty  of  its  parts,  to  standardize  the  responses  which  are  accept¬ 
able,  and  to  set  up  workable  norms  with  which  comparisons 
could  be  made — this  specifically  was  our  aim.  For  this  purpose 
we  have  chosen  an  opposites  test.  No  effort  has  been  made  to 
obtain  correlations  with  other  tests  and  no  claim  is  made  that 
the  test,  as  it  now  stands,  has  mentally  diagnostic  values  of  high 
importance  for  the  individual. 


» 


II. 

The  Results  Obtained  by  Previous  Investigators 

The  diversified  use  of  the  opposites  test,  with  its  different 
methods  of  presenting  and  scoring,  has  yielded  results  which, 
while  not  altogether  comparable,  are  not  without  interest. 

Dr.  Bonser1  in  an  investigation  of  the  reasoning  ability  of 
children  in  the  upper  division  of  the  fourth,  fifth,  and  sixth 
grades  of  the  public  schools  of  Passaic,  New  Jersey,  tested  three 
hundred  and  eighty-five  boys  and  three  hundred  and  seventy-two 
girls  within  the  period  of  February  13  to  27,  1906.  The  tests 
employed  were  designed  to  exercise  the  most  fundamental  phases 
of  reasoning  ability,  namely,  mathematical  judgment,  controlled 
association,  selective  judgment,  and  the  analytic  and  synthetic 
thinking  necessary  for  the  intellectual  interpretation  of  litera¬ 
ture. 

For  controlled  association,  three  types  of  tests  were  used. 
First,  in  two  sets  of  ten  sentences  presented,  a  significant  word 
was  omitted  from  each  sentence,  which  was  to  be  filled  in  by 
the  pupils.  Second,  two  other  sets  of  ten  sentences  were  given,  in 
each  of  which  two  significant  words  were  placed  one  above  the 
other.  The  pupils  were  instructed  to  draw  a  line  through  the 
wrong  word,  leaving  the  sentence  so  it  would  read  correctly. 
Third,  three  sets  of  twenty  words  each  were  given  to  the  pupils 
with  the  instructions  that  they  were  to  write  beside  each  re¬ 
spective  word  a  word  just  its  opposite  in  meaning. 

For  selective  judgment  two  types  of  tests  were  used.  The 
first  consisted  of  two  sets  of  two  series  each  of  ten  reasons  why 
some  given  fact  is  true,  some  of  which  are  correct,  the  others 
incorrect.  The  pupils  were  to  check  the  former.  Second,  there 
were  given  two  sets  of  three  series  each,  of  five  definitions  of  a 
given  thing  or  term,  some  of  which  were  correct,  others  incor- 

1  Bonser,  Frederick  G.,  “The  Reasoning  Ability  of  Children  of  the  Fourth, 
Fifth,  and  Sixth  School  Grades,”  Col.  Univ.  Coni.  Ed.,  37,  1906,  1-101. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


3 


rect  or  irrelevant.  The  pupils  were  to  choose  the  right  definition. 
In  the  table  below  these  tests  are  indicated  as  V  and  VI  respect¬ 
ively. 

For  literary  interpretation  the  pupils  were  asked  to  write  the 
meaning  of  two  stanzas  of  poetry. 

Below  is  shown  the  coefficients  of  correlation  obtained  by  cor¬ 
relating  each  test  with  the  combined  score  of  all  the  tests. 

TABLE  i 

Bonser’s  Correlations  of  Each  Test  with  the  Totals  for  All  Tests 

in  Reasoning  Ability. 


Opposites  and  Combined  Score  . 85 

Selective  Judgment  and  Combined  Score  (V) . 73 

Mathematical  Judgment  and  Combined  Score . 59 

Selective  Judgment  and  Combined  Score  (VI) . 58 

Controlled  Association  and  Combined  Score . 55 

Interpreting  Poems  and  Combined  Score  . 37 

Spelling  and  Combined  Score . 22 


Dr.  Bonser  has  arranged  for  comparison  separate  tables  for 
the  younger  and  the  older  group  in  each  grade.  He  finds  the 
younger  group  superior  in  the  opposites  test  for  every  grade,  but 
this  superiority  diminishes  as  the  years  in  school  increase.  This 
would  indicate,  he  thinks,  that  the  test  is  one  which  reveals  some 
sort  of  native  ability,  which  is  concealed  in  the  upper  grades 
where  the  test  is  relatively  simple. 

Miss  Norsworthy2  in  a  study  of  the  comparison  of  defective 
and  normal  children  found  that  the  defectives  were  farthest 
removed  from  normal  children  in  ability  to  deal  with  abstract 
data.  Of  the  137  cases  ranging  in  age  from  eight  years  up,  none 
of  the  defectives  surpassed  the  median  score  for  normal  children 
in  the  opposites  tests.  This  is  shown  in  the  following  table. 

Woolley  and  Fischer3  in  their  work  in  connection  with  the  in¬ 
dustrially  employed  children  in  Cincinnati  obtained  results  by  the 
use  of  opposites  tests  which  are  highly  significant.  Over  eight 
hundred  children  fourteen  years  of  age  were  given  the  following 

2  Norsworthy,  Naomi,  “The  Psychology  of  Mentally  Deficient  Children,” 
New  York,  Columbia  University,  1906. 

3  Woolley,  Helen  Thompson,  and  Fischer,  Charlotte  Rust,  “Mental  and 
Physical  Measurements  of  Working  Children,”  Psychol.  Monog.,  1914, 

XVIII,  77,  213-241. 


4 


MARIE  HACKL  MEANS 


TABLE  II 

Norsworthy’s  Percentages  of  Children  Scoring  above  2  P.  E.,  i  P.  E 
and  Median  for  Normal  Children  in  Series  of  Tests. 


%  above — 1 

%  above  — 2 

%  above  Med. 

P.  E.  (or 

P.  E.  (or 

for  ordinary 

lowest  25% 

lowest  9% 

children 

of  ordinary 

of  ordinary 

children) 

children) 

Height  . 

45 

61 

77 

Weight  . 

44 

66 

77 

Pulse  . 

49 

69 

86 

Temperature  . 

26 

59 

77 

Weight  Test . 

18 

28 

39 

A-T  Test . 

1 

14 

28 

Memory  of  Unrelated  Words 

6 

18 

27 

Dictation  . 

10 

10 

21 

Memory  of  Unrelated  Words 

5 

19 

30 

Part-Whole  Test  . 

9 

17 

27 

Genus-Species  Test  . 

9 

16 

17 

First  Opposite  Test . 

0 

0.9 

** 

a 

Second  Opposite  Test . 

0 

1 

7 

physical  and  mental  tests  upon  their  entrance  into 

the  industrial 

world :  Height,  Weight,  Visual  Acuity,  Auditory  Acuity,  Vital 
Capacity,  Strength  of  Hand,  Steadiness  of  Hand,  Tapping,  Card 
Sorting,  Cancellation  Test,  Memory  (Digits),  Substitution, 
Completion  of  Sentences,  Association  by  Opposites,  and  Puzzle 
Box  Test. 

A  year  later  six  hundred  and  seventy-nine  of  these  boys  and 
girls  were  re-tested. 

In  giving  the  opposites  test  one  of  the  eight  lists  of  twenty 
words  printed  one  under  another  was  presented  to  the  subject 
who  was  requested  to  write  beside  each  word  another  word  op¬ 
posite  in  meaning.  The  time  was  recorded  for  the  total  list  but 
the  scores  were  based  on  the  percentage  of  accuracy  alone.  Cred¬ 
its  of  one,  one-half,  or  zero  were  assigned  to  the  responses  given. 
Misspelled  words  were  given  full  credit  but  adjectives  written  in 
place  of  adverbs  received  only  half  credit. 

A  positive  correlation  was  obtained  between  the  school  grade 
completed  at  fourteen  years  of  age  and  the  ranking  in  every  one 
of  the  mental  tests.  The  general  order  is  as  follows :  Memory, 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


5 


Association  by  Opposites,  Sentence  Test,  Substitution  Test,  Can¬ 
cellation,  and  Puzzle  Box. 

The  public  school  children  were  found  to  be  superior  to  the 
parochial  school  group,  and  their  superiority  was  most  decided  in 
opposites  and  in  the  puzzle  box  tests,  the  two  tests  which,  accord¬ 
ing  to  the  investigators,  are  farthest  removed  from  a  relationship 
with  school  drill. 

‘‘There  is  a  marked  and  consistent  positive  correlation  with 
school  grade  in  this  test  (opposites)  for  both  sexes  and  at  both 
ages.  The  differences  from  grade  to  grade  are  so  large  and  so 
consistent  that  their  significance  cannot  be  questioned."4 

While  director  of  the  Department  of  Psychology  at  the  New 
York  State  Reformatory  for  Women,  at  Bedford  Hills,  Dr. 
Weidensall5  began  a  series  of  experiments  upon  the  women  de¬ 
tained  in  that  institution.  It  was  hoped  that  a  number  of  tests 
might  be  found  which  would  prove  prophetic  of  the  convicted 
woman’s  reformability  and  would  thus  eliminate  such  cases 
which,  because  of  inability  to  learn,  consumed  a  disproportionate 
amount  of  time. 

The  major  portion  of  the  monograph  deals  with  the  records 
of  the  criminal  woman  as  compared  with  the  norms  of  normal 
working  girls  fourteen  and  fifteen  years  of  age.  The  norms  were 
being  formulated  at  the  time  by  Dr.  Woolley6  in  her  work  as 
director  of  the  Bureau  of  Vocational  Guidance,  Cincinnati,  Ohio. 
Since  comparisons  were  to  be  made,  the  tests  which  were  being 
used  by  Dr.  Woolley  were  adopted  by  Dr.  Weidensall  and  given 
in  approximately  the  same  manner. 

The  women  tested  were  admitted  between  the  first  of  January 
and  the  end  of  October,  1913.  Two  hundred  and  eight  were 
committed  to  Bedford  during  that  period,  but  of  that  number 
only  a  hundred  were  tested.  Because  of  lack  of  facility  in  the 
English  language  of  twelve  foreigners,  the  percentile  tables  and 
curves  are  based  on  eighty-eight  records.  All  tests  were  given 

4  Woolley,  Helen  Thompson,  and  Fischer,  Charlotte  Rust,  Op.  cit.,  222. 

5  Weidensall,  Jean,  “The  Mentality  of  the  Criminal  Woman,”  Baltimore, 
Warwick  &  York,  1916,  3-266. 

6  Woolley,  Helen  Thompson,  and  Fischer,  Charlotte  Rust,  Op.  cit. 


6 


MARIE  HACKL  MEANS 


individually  and  during  the  first  two  weeks  of  confinement  while 
the  subject  was  still  in  quarantine. 

The  method  followed  in  connection  with  the  opposites  test 
was  to  present  a  list  of  twenty  words  and  to  record  the  total  time 
required  for  the  subject  to  write  the  opposites.  Results  were 
based  on  the  percent  of  accuracy,  as  it  was  found  that  the  cor¬ 
relation  between  rank  in  time  and  accuracy  in  this  test  was  as 
high  as  .83,  P.  E.  .029. 

All  identical  accuracy  scores  were  arranged  in  point  of  time 
of  performance  and  the  rankings  correlated  with  the  native 
ability  of  the  group  as  estimated  by  the  director  of  the  Industrial 
School  of  the  Reformatory  after  she  had  worked  with  these 
women  for  from  eighteen  months  to  two  years.  This  correlation 
was  -f-  .79,  P.  E.  .026,  and  was  obtained  by  the  formula 

6  2  d2 

n  (n2  —  1) 

This  correlation,  which  was  higher  than  that  for  any  other  test, 
would  probably  have  been  even  higher  had  it  not  been  necessary 
for  the  director  to  base  her  judgment  of  ten  of  the  women  on  the 
reports  of  matrons.  It  was  the  difference  in  rank  accorded  these 
ten  which  was  responsible  for  some  of  the  largest  variations 
from  the  rank  of  the  test. 

It  is  interesting  to  note  that  of  the  Bedford  eighty-eight  but 
thirty-nine  percent  attain  or  surpass  the  median  record  of  the 
working  girl  of  fifteen.  Dr.  Weidensall  feels  that  the  easy  op¬ 
posites  test  proved  the  most  reliable  of  all  tests  given,  for  clinical 
purposes. 

In  1912  Dr.  Benjamin  R.  Simpson7  selected  two  groups  as 
widely  different  in  intellectual  status  as  possible,  the  one  repres¬ 
ented  by  seventeen  professors  and  advanced  students  in  Columbia 
University,  the  other  by  inmates  of  charitable  institutions,  with 
the  exception  of  two  who  were  recognized  by  their  associates 
as  being  dull.  The  fifteen  tests,  which  were  given,  were  admin¬ 
istered  individually  and  in  the  same  order. 

The  hard  opposites  test  not  only  separated  the  two  groups  com- 

7  Simpson,  Benjamin  R.,  “Correlations  of  Mental  Abilities/'  Col.  Univ. 
Cont.  Ed.,  1912,  No.  53. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


7 


pletely  but  correlated  with  the  general  intelligence  of  the  good 
group,  as  estimated  by  the  members  which  composed  it,  more 
highly  than  any  of  the  other  tests.  The  individuals  of  the  good 
group  were  rated  in  order  of  merit  for  general  intelligence,  each 
by  the  rest  of  the  group,  four  years  after  the  tests  were  given. 
Two  rankings  made  by  the  experimenter  a  month  apart  were 
included.  The  judgments,  correlated  with  the  various  tests,  are 
as  follows : 

TABLE  III 

Simpson’s  Correlations  of  the  Estimated  Intelligence  of  his  Superior 
Group  with  the  Results  of  Eleven  Tests 


Estimated  Intelligence  and  Hard  Opposites . 96 

“  Ebbinghaus  Completion  Test . 89 

“  Memory  for  Words  . 93 

“  Memory  of  Passages  . 35 

Easy  Opposites  . 82 

“  “  “  Adding  . 72 

Learning  Pairs  . 34 

“  Completing  Words  .  1.00* 

“  “  “  “A”  Test  . 21 

“  Geometrical  Forms  . 07 

“  Drawing  Lengths  . . . — .20 


*This  coefficient,  according  to  Dr.  Simpson,  is  not  to  be  considered  reliable, 
since  the  reliability  coefficients  of  the  Completing  Words  test  in  the  Good 
group  is  only  .27. 

On  account  of  the  high  correlation  between  the  hard  opposites 
and  the  Ebbinghaus  Completion  Test,  Dr.  Simpson  feels  that  they 
test  the  same  mental  function,  namely  selective  thinking. 

Bronners  has  endeavored  to  determine  the  intellectual  status 
of  the  delinquent  girl  as  compared  with  the  intelligence  of  several 
other  groups  engaged  in  occupations  and  pursuits  which,  of  neces¬ 
sity,  required  varied  degrees  of  education  and  ability.  Thirty 
gills  living  at  Waverly  House,  a  detention  home  maintained  by 
the  New  York  Probation  Association,  composed  the  delinquent 
group.  The  college  group  contained  thirty-six  girls,  all  mem¬ 
bers  of  the  freshmen  and  sophomore  classes  of  Barnard  and 

Bronner,  Augusta.  F.,  A  Comparative  Study  of  the  Intelligence  of  De¬ 
linquent  Girls,”  Col.  Univ.  Cont.  Educ.,  1914,  No.  68. 


8 


MARIE  HACKL  MEANS 


Teachers’  College.  Thirty-four  Jewish  girls,  who  spent  their 
evenings  at  the  University  Settlement  or  the  Harlem  Branch  of 
the  Y.  W.  C.  A.,  engaging  in  educational  pursuits,  formed  the 
evening  class  group.  The  fourth  group  was  composed  of  twenty- 
nine  girls  who  had  never  engaged  in  any  wage-earning  occupa¬ 
tion  except  domestic  service.  None  of  these  girls  was  pursuing 
studies  whereby  she  hoped  to  prepare  herself  for  a  different  oc¬ 
cupation  nor  had  she  been  guilty  of  offenses  which  had  brought 
her  in  conflict  with  the  law. 

In  addition  to  an  ethical  discrimination  test,  those  tests  em¬ 
ployed  comprise  the  Easy  Opposites,  the  Hard  Opposites,  the 
Memory  of  Words,  the  Memory  of  Passages,  and  the  Ebbinghaus 
Completion  test.  These  tests  were  the  same  as  those  used  by  Dr. 
Benjamin  Simpson  in  his  study  of  Correlations  of  Mental  Abili¬ 
ties .  In  the  following  table,  which  is  a  reorganization  of  five 
tables  presented  by  Bronner,  D  represents  the  delinquent  group, 
C  the  college  group,  E  the  evening  class  group,  and  S  the  domestic 
service  group.  These  data  show,  as  Bronner  states,  that  the 

TABLE  IV 

Comparisons  of  the  Four  Groups 
Easy  Opposites 

%  of  D  reaching  the  25  percentile  of  the  C  group.... 

%  of  E  reaching  the  25  percentile  of  the  C  group.... 

%  of  S  reaching  the  25  percentile  of  the  C  group.... 

Hard  Opposites 

%  of  D  reaching  the  25  percentile  of  the  C  group.... 

%  of  E  reaching  the  25  percentile  of  the  C  group _ 

%  of  S  reaching  the  25  percentile  of  the  C  group _ 

Memory  of  Unrelated  Words 
%  of  D  reaching  the  25  percentile  of  the  C  group.... 

%  of  E  reaching  the  25  percentile  of  the  C  group.... 

%  of  S  reaching  the  25  percentile  of  the  C  group _ 

Memory  of  Passages 

%  of  D  reaching  the  25  percentile  of  the  C  group.... 

%  of  E  reaching  the  25  percentile  of  the  C  group.... 

%  of  S  reaching  the  25  percentile  of  the  C  group _ 

Ebbinghaus  Completion  Test 
%  of  D  reaching  the  25  percentile  of  the  C  group.... 

%  of  E  reaching  the  25  percentile  of  the  C  group.... 

%  of  S  reaching  the  25  percentile  of  the  C  group.... 


3-3% 

15.0% 

7.0% 

0.0% 

12.0% 

0.0% 

20.0% 

59-0% 

7.0% 

10.0% 

15.0% 

3.0% 

6.6% 

9-0% 

3-o% 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


9 


college  girls  excel  in  all  the  tests.  The  delinquent  girl  is  quite  as 
capable  as  the  domestic  service  girl,  while  both  are  surpassed  by 
the  girls  attending  evening  classes. 

The  superiority  of  the  evening  class  girls  over  the  other  two 
groups  displayed  in  each  test,  cannot  be  attributed  to  more  favor¬ 
able  educational  advantages  for  none  of  the  members  had  attended 
high  school,  whereas  of  the  delinquent  group,  four  had  reached 
the  eighth  grade,  two  the  first  year  of  high  school,  two  the  second 
year,  one  was  graduated,  and  one  had  attended  a  normal  school. 

The  Hard  Opposites  Test  separated  the  college  group  from  the 
others  almost  entirely.  Next  in  order  comes  the  Ebbinghaus 
Completion  Test,  with  the  Easy  Opposites  ranking  third.  It  is 
not  without  interest  that  Dr.  Simpson  found  the  Hard  Opposites 
separated  his  good  group  entirely  from  his  poor  group  and  the 
Easy  Opposites  in  this  respect  surpassed  the  Ebbinghaus  Test. 

Let  us  now  examine  the  record  of  the  Opposites  Test  when 
applied  to  the  highly  selected  group,  represented  by  the  college 
student.  In  1914  Dr.  Kitson9  began  a  series  of  tests  upon  the 
freshmen  in  the  College  of  Commerce  and  Administration  of  the 
University  of  Chicago.  One  of  the  sixteen  tests  used  was  an 
opposites  test.  Two  lists  of  twenty  words  each  were  presented 
and  the  time  recorded  for  the  subject  to  call  the  opposites  of  each 
list.  Five  points  were  deducted  for  each  wrong  word  or  for 
failure  to  respond  within  fifteen  seconds,  from  the  one  hundred 
points  allowed  each  list.  The  final  score  for  the  individual  was 
obtained  by  dividing  the  time  score  by  the  accuracy  score. 

The  stimuli  were  of  such  little  difficulty  that  they  measured  for 
the  most  part  merely  speed  of  association.  Almost  half  of  the 
subjects  obtained  an  accuracy  score  of  one  hundred  in  each  list. 
Of  forty  students,  thirty-one  scored  one  hundred  in  accuracy  in 
the  easy  list,  and  twenty-one  scored  a  hundred  in  the  hard  list. 

In  his  results  Kitson  has  included  the  following  table  of  cor¬ 
relations. 

Of  the  fifteen  tests  used  by  Kitson,  by  computing  the  multiple 
and  the  partial  correlations  and  the  regression  coefficients, 

9  Kitson,  H.  D.,  “The  Scientific  Study  of  College  Students,”  Psychol. 
Monog.,  1917,  23. 


10 


MARIE  HACKL  MEANS 


TABLE  V 

Correlations  of  Standings  in  Each  Test  with  Standings  in  the  Net  Score 

(Method  of  Rank  Differences) 

Correlations  P.  E. 


Logical  Material  Seen  (Deferred)  .  .60  .07 

Opposites  .  -53  -°8 

Hard  Directions  (Printed)  .  -49  -°8 

Objects  Seen  .  .48  -°8 

Loss  in  Logical  Material  Seen  .  .4 7  -09 

Logical  Material  Heard  (Deferred)  .  -45  -°9 

Word  Building  .  -45  -09 

Loss  in  Logical  Material  Heard  .  .43  *09 

Sentence  Building  .  -42  -09 

Constant  Increment  .  .38  .10 

Business  Ingenuity  .  .33  .10 

Logical  Material  Seen  (Immediate)  .  .29  .10 

Numbers  Heard  .  .27  .10 

Hard  Directions  (Oral)  .  .23  .11 

Logical  Material  Heard  (Immediate)  .  .23  .11 

Number  Checking  .  .18  .11 


Rosenow10  concludes  that  five  of  the  tests  carry  all  the  meaning 
with  reference  to  school  marks  and  hence  all  diagnostic  value. 
In  the  table  below  the  writer  has  rearranged  the  results  of 
Rosenow's  investigation.  From  it  one  learns  that  the  probability 
is  1300  :  1  that  the  Logical  Memory  is  significant. 

\ 

TABLE  VI 

Rearrangement  of  the  Results  of  Rosenow’s  Investigation 

Probability  is  1300  :  1  that  the  Logical  Memory  Test  is  Significant. 

25  :  1  that  the  Constant  Increment  Test  is  Significant. 

30  :  1  that  the  Sentence  Building  Test  is  Significant. 

23  :  1  that  Auditory  Presentation  is  Superior  to  Visual. 

140  :  1  that  the  Loss  or  Gain  in  Logical  Auditory  Memory 
is  significant. 

that  the  Hard  Directions  Test  has  Negative  Significance. 

that  the  Objects  Seen  Test  has  Negative  Significance. 

The  remaining  tests,  including  the  Opposites,  have  no  diag¬ 
nostic  value. 

In  view  of  the  fact  that  the  stimuli  used  in  the  Opposites  Test 
were  the  Woodworth  and  Wells  standardized  list,  Rosenow’s  con- 

10  Rosenow,  Curt,  “The  Analysis  of  Mental  Functions,”  Psychol.  Monog., 
1917,  24. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


ii 


elusions  are  not  surprising.  The  words  were  of  so  little  difficulty 
that  they  tested  only  speed  of  association. 

King  and  M’Crory11  in  testing  two  hundred  and  seventy-six 
women  and  two  hundred  and  sixty-eight  men  in  the  freshmen 
class  at  the  University  of  Iowa,  applied  seven  tests,  including 
both  easy  and  hard  opposites.  The  following  table  indicates  the 
correlations  which  were  obtained  between  the  Opposites  and 
other  tests  as  well  as  university  grades.  The  Opposites  Test, 
they  find,  correlates  most  highly  of  any  of  the  tests  with  the 
university  marks  of  both  men  and  women. 

TABLE  VII 

Correlations  of  Opposites  with  Other  Tests  and  University  Grades 

Correlations 
Women  Men 


Opposites  and  Completion . 31  -79 

“  “  Arithmetic  (Speed)  . 03  Neg. 

“  “  Arithmetic  (Accuracy)  . 01  Neg. 

“  “  Analogies  . 52  .77 

“  “  Information  . 24  .56 

“  Visual  Imagery  . 07  .56 

“  “  Logical  Memory  . 32  .38 

“  Test  Average  . 51  .88 

“  University  Grades  . 45  .84 


Dr.  King12  is  of  the  opinion  that  the  Opposites  Test,  if  thor¬ 
oughly  standardized  and  used  in  conjunction  with  other  tests, 
will  yield  results  of  great  importance. 

In  the  Spring  of  1916,  while  the  writer  was  yet  engaged  in  the 
selection  of  appropriate  stimuli  for  the  present  investigation,  an 
opportunity13  presented  itself  of  correlating  the  grades  of  seventy- 
three  students  in  psychology  with  the  results  secured  by  the  use 
of  the  Opposites  Test.  An  effort  had  been  made  to  keep  the  two 
lists  composed  of  one  hundred  and  fifty  words  each,  as  nearly 

11  King,  Irving,  and  M’Crory,  J.  L.,  “Freshmen  Tests  at  the  State  Univer¬ 
sity  of  Iowa,”  Jour.  Educ.  Psychol.,  1918,  IX,  32-46. 

12  King,  Irving,  and  Gold,  Hugo,  “A  Tentative  Standardization  of  Certain 
Opposites  Tests,”  Jour.  Educ.  Psychol.,  1916,  VII,  459-482. 

13  Dr.  Edward  K.  Strong,  at  that  time  Professor  of  Psychology  at  George 
Peabody  College  for  Teachers,  kindly  furnished  these  data.  The  grades 
were  compiled  from  the  records  of  fourteen  tests  given  during  the  quarter. 


12 


MARIE  HACKL  MEANS 


equal  as  possible.  The  two  lists  are  given  below.  The  words 
are  arranged  in  random  order.  Those  words  which  had  not  been 
used  by  previous  investigators  are  in  italics. 

List  i. — Defective,  Late,  Sinful,  Easy,  Hilly,  Superior,  Girl, 
White,  Cool,  Large,  Evil,  Queen,  Deep,  True,  Public,  Sink, 
Future,  Adroit,  Dangerous,  Day,  Ugly,  Quick,  Poor,  Diligent, 
Wicked,  Round,  Ceiling,  Broken,  Gentle,  Vague,  Brief,  Ani¬ 
mated,  Slovenly,  Dim,  Out,  Rude,  Lazy,  Injurious,  Conservative, 
Wet,  Asleep,  Stingy,  Fertile,  Wise,  Calm,  Tardy,  Hinder,  Re¬ 
spect,  Big,  Gain,  Great,  Profit,  Young,  Few,  Summer,  Above, 
Glad,  Masculine,  Remember,  Off,  Beginning,  Love,  Straight, 
War,  Joy,  Naked,  Pride,  Apart,  Brave,  Noisy,  Fickle,  Create, 
Wild,  Despondent,  Frequently,  Timid,  Hollow,  Belief,  Bad,  Up, 
Sick,  Empty,  Strong,  Inside,  Front,  After,  Broad,  Sharp,  Sweet, 
Succeed,  Add,  Happy,  Raise,  Aristocratic,  East,  Short,  Thick, 
Result,  Rare,  Stale,  North,  Hostile,  Laugh,  Obnoxious,  Ex¬ 
pensive,  Near,  Join,  Hot,  Forcible,  Preserve,  Strict,  Handsome, 
Friend,  Miser,  Exciting,  Rough,  Brother,  Light,  Careful,  Push, 
Haughty,  Impoverish,  Busy,  Much,  Graceful,  Ocean,  Precise, 
Barbarous,  Ignorant,  Reckless,  Odd,  Victorious,  Repulsion,  Per¬ 
mit,  Positive ,  Pessimistic ,  Extravagant ,  Durable,  Analytical,  Par¬ 
simony,  Orthodoxy,  Acute,  Exoteric,  Antonym,  Dorsal,  Longi¬ 
tude,  Divide,  Infinity,  Dynamic,  Posterior. 

List  II. — Gay,  Foolish,  Drop,  Giving,  Cloudy,  Blunt,  Beauti¬ 
ful,  Backwards,  Well,  Top,  Success,  Soft,  New,  Refined,  Weary, 
Spend,  Break,  Male,  Country,  Dark,  Weak,  Black,  Disastrous, 
Rigid,  Elation,  Hindrance,  Savage,  Degrade,  Ripe,  Shaky,  Sepa¬ 
rate,  Liquid,  Sell,  Honest,  Difficult,  Dirty,  Wrong,  Winter,  Help¬ 
less,  Obscure,  Expand,  Insignificant,  Sleepy,  Sad,  Little,  Enemy, 
Open,  Yours,  Yes,  Conservative,  Soothing,  Doubtful,  Sacred, 
Sure,  Reveal,  Stupid,  Motion,  Sickly,  Slowness,  Outside,  Same, 
Cowardly,  Float,  Foreign,  Strength,  Sane,  Level,  Simple,  Many, 
Lost,  Something,  Sour,  Enrage,  Serious,  Long,  In,  Take,  Tight, 
Prompt,  Patient,  Permanent,  Genuine,  Morning,  Smooth,  Heavy, 
Full,  Grand,  Humility,  Tall,  Over,  First,  Strife,  Follower,  Hold, 
Proficient,  Vertical,  Shallow,  Absent,  Rapid,  Rich,  Purity,  Lo¬ 
quacious,  Imaginary,  Silly,  Increase,  Wider,  Nowhere,  Upper, 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


13 


Woman,  Generous,  Careless,  Scarce,  Height,  Always,  Wife,  Best, 
Below,  Thin,  High,  Early,  Suave,  Lack,  Advance,  Harmonious, 
Bless,  Both,  Cruel,  Cheap,  Ancient,  Less,  Forget,  Come,  Slow, 
Good,  Negative ,  Optimistic ,  Economical ,  Perishable,  Synthetical, 
Prodigality,  Heterodoxy,  Chronic,  Esoteric,  Synonym,  Ventral, 
Latitude ,  Multiply,  Zero ,  Static,  Anterior. 

The  test  was  given  individually  and  orally.  The  experimenter 
read  the  stimulus  and  recorded  the  response  of  the  subjects  as 
well  as  his  reaction  time.  The  accuracy  score  in  percentage  and 
the  time  score  arrived  at  by  computing  the  median  time  of  all 
reactions  were  tabulated  for  each  student. 

In  grading  these  papers,  credits  of  one,  one-half,  or  zero  were 
assigned  to  the  responses.  What  credit  a  word  deserved  was 
determined  solely  by  the  writer,  as  she  was  not  yet  ready  to 
standardize  the  responses  for  so  bulky  a  list. 

In  order  to  allow  equal  credit  for  time  and  accuracy  the  follow¬ 
ing  formula  was  used : 

Dt  Da 

Individual  score  = - 1 - in  which 

OT  QA 

2 

Da  =  the  deviation  of  the  individual  accuracy  scores  from 
the  accuracy  scores  of  the  group. 

Dt  =  the  deviation  of  the  individual  time  score  from  the 
time  score  of  the  group.  Since  signs  were  regarded, 
where  an  individual  had  a  lower  time  score  than  the 
group  his  deviation  was  positive. 

QA=  the  quartile  of  the  accuracy  scores  of  the  group. 

QT  =  the  quartile  of  the  time  scores  of  the  group. 

The  coefficient  of  correlation  obtained  by  the  method  of  rank 
differences  was  .54,  P.  E.  =  .08.  Speed  and  accuracy  were 
likewise  correlated,  giving  a  coefficient  of  .33,  P.  E.  =  .10.  This 
coefficient  would  hardly  justify  the  statement  that,  in  so  far  as 
this  particular  test  is  concerned,  either  accuracy  or  speed  could  be 
disregaded  without  affecting  the  rankings  in  the  test. 

In  Figure  I  is  shown  the  relative  positions  of  the  several  mem¬ 
bers  of  the  class  in  psychology  and  in  the  Opposites  Test.  The 


14 


MARIE  HACKL  MEANS 


vertical  line  indicates  the  rank  in  the  test,  the  horizontal  line,  the 
rank  in  psychology.  A  glance  at  the  figure  will  show  that  on  the 
whole  those  good  in  the  test  were  good  in  psychology.  The 
greatest  exception  is  to  be  found  in  the  position  of  the  student 
indicated  by  the  figure  5  in  the  graph.  This  woman  of  splendid 
ability  was  permitted  by  her  physician  to  remain  in  school  only 
on  the  condition  that  she  would  do  the  least  possible  amount  of 
work  to  secure  credit  for  the  course  in  which  she  was  enrolled. 
Case  4,  a  widow,  had  been  out  of  school  and  school  work  for  a 
number  of  years.  The  increased  cost  of  living  had  forced  her 
to  return.  Case  3  had  registered  in  psychology  because  it  was 
one  of  the  required  courses. 


Figure  i 


The  relations  between  rank  based  on  psychology  grades  and  rank  based 
on  the  Opposites  Test  score  are  shown  here.  Each  circle  indicates  the  posi¬ 
tion  of  one  of  the  seventy-three  college  students.  Units  on  the  vertical  line 
denote  rank  in  the  test;  horizontal  units  are  ranks  in  psychology. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


15 


On  the  other  hand,  it  seems,  one  could  predict  with  less  cer¬ 
tainty  the  standing  in  the  class  of  those  students  who  are  some¬ 
what  poor  in  the  test.  This  may  be  due  in  part  to  the  inadequacy 
of  the  test  itself,  in  part  to  the  inequality  of  interest  in  psy¬ 
chology.  Cases  1  and  2  are  typical  of  those  students  whose  class 
work  represents  their  best  effort.  These  cases  are  selected  be¬ 
cause  they  are  the  only  ones  falling  within  the  writer’s  experience. 
A  careful  analysis  of  the  individual  records,  if  possible,  might 
reveal  results  of  significance. 

While  the  results  indicate  that  the  test  as  used  by  the  different 
investigators  reveals  some  sort  of  native  ability,  just  what  this 
ability  is  and  to  what  extent  it  is  revealed  remains  to  be  seen. 
Certainly,  results  of  various  experimenters  up  to  date  are  not 
comparable,  nor  will  they  be  until  identically  the  same  test  is 
given  with  the  same  method  of  presenting  and  scoring. 


III. 

Previous  Efforts  toward  Standardization 

The  first  attempt  to  meet  this  need  of  standardization  was 
made  by  Woodworth  and  Wells1  in  1911.  After  testing  six  sub¬ 
jects  with  a  long  list  of  words,  forty  words  were  selected,  which 
in  turn  were  tried  with  forty  other  subjects.  A  few  more  words 
were  substituted  from  tests  of  a  few  subjects,  then  the  revised  col¬ 
lection  was  tried  with  thirteen  fresh  subjects  (all  college  and 
graduate  students)  and  a  few  more  minor  corrections  introduced. 

In  the  experiment  the  four  lists  of  ten  stimulus  words  were 
presented  visually  but  the  time  of  the  single  reaction  as  well  as 
the  total  time  was  roughly  taken. 

An  attempt  was  made  to  have  the  halves  of  the  list  of  equal 
difficulty,  and  since  it  was  found  impossible  to  prepare  a  list  of 
twenty  stimuli  of  equal  difficulty,  the  words  were  combined  in 
pairs,  “so  that  pairs  should  be  of  equal  difficulty,  as  judged  by 
the  sum  of  the  reaction  times  to  the  two  members  of  each  pair.” 
One  pair,  for  instance,  consisted  of  the  hardest  and  the  easiest 
word  in  the  list  and  another  pair  of  two  words  of  medium  dif¬ 
ficulty. 

In  the  event  that  the  test  might  be  given  with  a  time  limit,  the 
authors  have  placed  the  words  of  medium  difficulty  in  the  list 
where  most  of  the  subjects  will  be  stopped,  namely,  from  about 
the  eighth  to  the  sixteenth  word.  “If  then  the  time  limit  is  so 
chosen  that  the  great  majority  of  subjects  shall  be  stopped  in 
this  list,  the  separate  words  may,  without  much  error  on  the  aver¬ 
age,  be  counted  as  equal  units.” 

The  test  as  given  by  Woodworth  and  Wells  is  scored  only  on 
the  basis  of  time.  The  words  were  so  selected  that  none  of  the 
subjects  could  fail  to  have  a  perfect  accuracy  score.  As  such,  it 
is,  as  the  authors  intended,  merely  a  test  of  the  speed  of  associa¬ 
tion. 

1Woodworth,  R.  S.,  and  Wells,  F.  L.,  “Association  Tests,”  Psychol. 
Monog.,  1911,  XIII. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


17 


In  1916  a  further  attempt  was  made  by  King  and  Gold2  to 
standardize  the  test.  The  four  lists  of  twenty  easy  words  and  the 
four  lists  of  twenty  hard  words,  as  used  by  Simpson,3  were  pre¬ 
sented  individually  to  nine  faculty  members,  twenty-three  gradu- 
are  students,  forty-seven  seniors,  and  twenty-one  juniors  of  the 
Departments  of  Education  and  Psychology  in  the  University  of 
Iowa. 

The  subjects  were  instructed  to  take  one  list  at  a  time,  go  down 
the  list  and  name  orally  the  opposite  of  each  word  in  rapid  suc¬ 
cession  and  to  avoid  wherever  possible  prefixing  a  syllable  to  the 
stimulus  word.  The  time  was  recorded  for  each  list  separately 
and  the  responses  were  taken  down  in  shorthand. 

The  easy  and  hard  opposites  were  scored  differently.  In  the 
case  of  the  easy  opposites  a  credit  of  one,  one-half,  or  zero  was 
given  to  responses,  and  the  penalty  for  an  omitted  word  was  four 
seconds.  To  the  various  responses  to  the  hard  opposites,  credits 
of  one,  two-thirds,  one-third,  and  zero  were  assigned,  and  an 
omitted  word  or  incorrect  word  was  penalized  eighteen  seconds. 
Regardless  of  the  fact  that  the  words  were  of  varying  difficulty, 
the  penalty  remained  the  same,  namely,  four  seconds  for  easy 
words  and  eighteen  seconds  for  hard  words. 

On  the  basis  of  these  results  (one  hundred  records  for  each 
word)  the  percent  of  failures  for  each  word  was  computed.  For 
each  stimulus  is  recorded  the  value  in  terms  of  accuracy  per¬ 
centage  as  well  as  the  acceptable  responses  and  the  frequency  of 
each. 

In  the  work  of  Harry  A.  Greene4  we  find  the  first  attempt  to 
assign  to  each  word  a  point  value  based  on  its  relative  difficulty. 
Greene  presented  the  stimuli  used  by  King  and  Gold5  to  two 
groups  of  freshmen  in  the  University  of  Iowa  in  the  fall  of  1916- 
1917.  Nine  hundred  and  ninety  individuals  were  tested  by  the 
first  half  of  the  list  and  seven  hundred  and  ten  by  the  second 

2  King,  Irving,  and  Gold,  Hugo,  “A  Tentative  Standardization  of  Certain 
Opposites  Tests,”  Jour.  Educ.  Psychol.,  1916,  VII,  459-482. 

3  Simpson,  Benjamin  R.,  Op.  cit. 

4  Greene,  Harry  A.,  “A  Standardization  of  Certain  Opposites  Tests,”  Jour. 
Educ.  Psychol.,  1918,  IX,  559-566. 

5  King,  Irving^  and  Gold,  Hugo,  Op.  cit. 


i8 


MARIE  HACKL  MEANS 


half.  The  words  were  graded  by  the  King  and  Gold  standardiza¬ 
tion  with  the  exception  that  words  given  two-thirds  credit  by  them 
were  given  one-half  credit  and  those  receiving  one-third  credit 
were  scored  wrong. 

The  percentage  of  failures  was  then  determined  by  allowing 
a  value  of  one  unit  for  correct  responses  and  one-half  for  half- 
correct  responses  and  subtracting  this  total  from  nine  hundred 
and  ninety.  In  the  case  of  those  words  from  the  last  half  of  the 
list  for  which  there  were  only  seven  hundred  and  ten  responses 
available,  these  totals  were  increased  in  the  proportion  of  nine 
hundred  and  ninety  to  seven  hundred  and  ten,  thus  making  all 
the  words  in  the  list  comparable. 

By  reading  directly  from  the  table  based  upon  the  area  of  the 
probability  curve  and  assuming  that  the  base  line  is  broken  ar¬ 
bitrarily  at  +3  sigma,  the  percentage  scores  were  changed  into 
percentile  values.  These  values  were  then  totaled  and  each  value 
in  turn  divided  by  the  total,  thus  converting  the  percentile  values 
into  relative  point  values.  The  points  were  based  on  accuracy 
alone,  no  account  being  taken  of  time. 


IV. 


The  Selection  of  Stimuli  for  the 
Present  Investigation 

At  the  very  outset,  the  writer  was  confronted  with  the  problem 
of  the  method  in  which  the  test  was  to  be  presented.  Chiefly 
because  of  the  opportunity  it  afforded  of  studying  each  word 
individually,  the  oral  method  was  chosen.  Each  word  thus  be¬ 
came  a  unit.  The  list  could  be  lengthened  or  shortened  at  will 
without  influencing  to  any  appreciable  extent  the  words  retained. 
Let  it  be  held  in  mind  that  at  this  point  our  interest  was  two-fold, 
namely,  the  selection  of  suitable  stimuli  and  the  computation  of 
the  value  in  points  to  be  assigned  to  each  word. 

To  have  given  the  test  in  groups  would  have  simplified  the 
scoring  and  would  have  dispensed  with  much  effort,  but  the  at- 
tendent  disadvantages  made  it  necessary  to  abandon  this  method 
as  a  possible  choice. 

In  the  first  place,  it  was  intended  that  the  test  should  be  more 
difficult  than  previous  tests  of  this  character.  This  meant  the 
substitution  of  entirely  new  stimuli,  stimuli  to  be  acquired  only 
by  the  slow  process  of  trial  and  error.  Furthermore  it  was  en¬ 
cumbent  upon  the  writer,  due  to  the  small  number  of  subjects 
at  her  disposal,  to  avoid  a  method  which  would  involve  discard¬ 
ing  the  entire  record  of  a  subject  because  of  the  use  of  one  or 
more  undesirable  stimuli.  As  given,  each  word  was  a  problem 
apart  from  every  other  word  and  only  the  records  of  those  words 
not  in  the  list  as  finally  decided  upon,  were  thrown  into  the 
discard. 

If  the  group  method  is  to  be  employed,  a  decision  must  be 
made  as  to  whether  time,  accuracy,  or  a  combination  of  both  will 
be  considered  a  measure  of  the  difficulty  of  the  word.  Time 
considered  alone,  the  test  becomes  one  largely  of  speed  of  asso¬ 
ciation.  If  the  test  is  to  be  given  with  a  time  limit,  the  individual 
responses  lose  their  identity  as  units  and  the  percentage  of  failures 


20 


MARIE  HACKL  MEANS 


per  stimulus  will  vary  with  the  time  allowed.  For  instance,  if 
the  time  limit  is  reduced  from  six  to  three  minutes,  the  percentage 
of  failures  will  vary  considerably,  particularly  in  the  case  of 
the  more  difficult  words  which  should  be  placed  at  the  end  of  the 
list.  Thus  one  can  see  how  a  shortened  time  limit  might  give 
identical  accuracy  scores  with  words  of  unequal  difficulty. 

To  illustrate  the  effect  of  varying  the  time  limit  not  only  upon 
the  percentage  of  failures  but  also  upon  the  nature  of  the  re¬ 
sponses  themselves,  let  us  consider  the  responses  to  the  word 
“sacred”  given,  first,  by  a  group  of  fifty-six  students  tested  orally 
and  individually,  allowing  ten  seconds  for  the  response,  second, 
by  another  group  of  fifty-six  students  similarly  tested,  and  third, 
by  a  group  of  fifty-six  who  were  allowed  only  six  minutes  to 
write  the  opposites  to  the  list  of  sixty-eight  words  eventually  re¬ 
tained.  In  the  table  below  is  shown  the  number  of  failures  and 
the  exact  responses  given  by  the  three  groups  of  fifty-six.  Notice 
the  greater  number  of  failures  in  Group  III  and  also  the  homo¬ 
geneity  of  responses.  This  may  be  explained  by  the  fact  that  the 
word  is  not  reached  in  the  list  or  that  uncertain  words  are  passed 
up  for  those  in  which  the  subject  feels  confident  of  success. 

On  the  other  hand  to  lengthen  the  time  limit  considerably, 
while  it  would  give  more  nearly  correct  accuracy  scores,  would 
be  unfair  to  the  individuals  tested,  if  these  same  records  were  to 
be  used  as  the  basis  for  norms.  With  a  lengthened  time  limit, 
students  would  be  able  to  complete  the  test  at  different  intervals. 
Evidently  of  two  students,  both  of  whom  make  perfect  scores, 
the  one  finishing  the  work  in  the  shorter  period  is  the  more  ef¬ 
ficient.  But  we  should  have  no  measure  of  this  efficiency. 

To  give  the  test  as  a  group  test  without  taking  into  considera¬ 
tion  the  time  consumed,  would  seem  to  imply  either  that  addi¬ 
tional  time  would  fail  to  increase  the  accuracy  percentage,  or 
that  without  exception  the  more  difficult  a  word,  the  longer  the 
time  required  to  think  of  an  opposite.  The  former  assumption  is 
probably  true  within  certain  limits.  It  is  conceivable  that  a  time 
limit  might  be  selected  beyond  which  there  would  be  no  improve¬ 
ment,  but  it  is  obvious  that  the  time  limit  might  be  so  shortened 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


21 


TABLE  VIII 

Showing  the  Different  Responses  Given  as  an  Opposite  to  the  Word 
“Sacred”  when  the  Time  Limit  is  Changed 


Number  of  Times  Each  Response 
is  given  by  the  Several  Groups 


Different  Responses 

Given  by  the  Groups 

Group  I 

Group  II 

Group  III 

Common  . 

I 

i 

Secular  . 

2 

4 

Profane . 

.  5 

7 

4 

Sinful  . 

2 

I 

Worldly  . 

2 

I 

Unholy  . 

.  9 

I 

2 

Irreverent  . 

.  2 

3 

I 

Wicked  . 

.  2 

2 

I 

Sacrilegious  . 

.  3 

7 

2 

Unsacred  . 

.  8 

5 

0 

Vulgar  . 

.  i 

3 

0 

Public  . 

.  2 

I 

0 

Mean  . 

i 

0 

Ungodliness  . 

.  0 

i 

0 

Heathen  . 

I 

0 

Sacrilege  . 

I 

0 

Obnoxious  . 

.  I 

0 

0 

Ungodly  . 

.  I 

0 

0 

Unnoticed  . 

.  I 

0 

0 

Idolatrous  . 

.  I 

0 

0 

Irreligious  . . 

0 

0 

Infidel  . 

0 

o 

Hypocritical  . 

0 

o 

Failure  to  Respond . 

.  15 

16 

39 

Number  of  Different 

Responses  per  Group.... 

16 

9 

that  the  accuracy  scores  would  be  greatly  affected.  In  general 
it  is  true  that  words  with  a  high  accuracy  score  have  a  corre¬ 
spondingly  low  time  score,  but  a  glance  at  Tables  XI  and  XII 
will  show  that  such  is  not  always  the  case,  for  words  with  differ¬ 
ent  accuracy  scores  may  have  identical  time  scores  and  words 
with  identical  accuracy  scores  have  different  time  scores.  How¬ 
ever,  the  correlation  between  the  two  is  undoubtedly  high. 

To  give  the  test  individually  and  record  the  time  for  the  total 
list  has  all  the  disadvantages  in  point  of  labor  and  none  of  the 
advantages  that  accrue  when  the  test  is  given  individually  and 
orally. 


22 


MARIE  HACKL  MEANS 


Having  determined  the  procedure  to  be  followed,  the  writer 
began  in  the  fall  of  1915  the  task  of  selecting  suitable  stimuli 
for  the  present  investigation.  After  eleven  records  were  obtained 
with  a  list  containing  the  three  hundred  and  twenty-three  words 
used  by  previous  experimenters  and  forty  new  words,  the  list 
was  divided  and  during  the  remainder  of  the  year  two  lists  were 
used  which  consisted  on  the  average  of  about  one  hundred  and 
fifty  words  each.  From  time  to  time  words  found  inadequate 
were  dropped  and  others  substituted  as  they  occurred  to  the 
writer.  Hence  the  inequality  in  the  number  of  subjects  tested 
with  a  given  word.  In  this  manner  one  hundred  and  thirty-nine 
students  at  George  Peabody  College  for  Teachers  were  tested 
during  the  school  year  1915-1916. 

Commonly  misunderstood  words  were  weeded  out  as  were 
those  with  an  accuracy  score  of  one  hundred  percent.  Likewise 
attempts  were  made  to  discard  those  words  which  had  an  oppo¬ 
site  formed  by  adding  the  prefix  “un”  in  frequent  and  reputable 
use.  Meanwhile  original  words  were  subjected  to  the  same  pro¬ 
cess  of  examination  and  elimination. 

The  point  of  interest  was  the  individual  word,  but  as  these 
data  were  to  be  used  in  another  connection  care  was  taken  that 
the  conditions  remain  constant  throughout.  Since  each  test  was 
given  individually,  it  was  an  easy  matter  to  change  the  order  of 
the  stimuli  and  thus  avoid  practice  effects.  The  reaction  time  in 
fifths  of  a  second  was  recorded  with  a  stop-watch  along  with  the 
response.  The  following  directions  were  read  to  each  subject: 

“As  soon  as  I  read  a  word  you  are  to  give  me  the  best  opposite 
you  can  think  of.  For  instance,  if  I  read  'black’  you  are  to  say 
'white/  Do  not  give  me  phrases,  nor  words  beginning  with  the 
prefix  'non.’  The  word  you  give  must  belong  to  the  same  part 
of  speech  as  the  word  in  the  list.  Your  time  will  be  recorded, 
so  answer  each  word  as  quickly  as  possible.  Under  no  circum¬ 
stances  will  I  allow  you  over  ten  seconds  for  a  word.  Let  me 
show  you  how  long  ten  seconds  really  is.  (Experimenter  illus¬ 
trates  with  stop-watch.)  Remember  in  each  case  to  respond  as 
quickly  as  possible.  Do  you  understand  what  you  are  to  do?” 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


23 


In  the  fall  of  19181  the  work  was  resumed.  It  was  decided 
expedient  to  use  only  one  list  as  the  number  of  possible  subjects 
to  be  obtained  at  Peabody  was  small.  Two  lists  would  of  course 
cut  the  records  for  each  word  in  two.  Besides  the  second  list 
contained  for  the  most  part  opposites  of  the  first.  Of  the  one 
hundred  words  with  which  the  work  was  begun  in  the  fall,  thirty- 
two  were  dropped,  leaving  the  sixty-eight  which  comprise  the 
list  as  it  now  stands.  These  words,  arranged  in  the  ascending 
order  of  difficulty,  are  given  below.  The  original  words  are 
given  in  italics.  Full,  Negative •,  After,  Dim,  Blunt,  Success, 
Pessimistic,  Joy,  Public,  Profit,  Spend,  Always,  Graceful, 
Strength,  Ancient,  Expand,  Barbarous,  Hinder,  Despondent, 
Vague,  Fertile,  Doubtful,  Injurious,  Busy,  Abstract,  Advance, 
Foreign,  Create,  Simple,  Extravagant,  Aristocratic,  Rare,  Dan¬ 
gerous,  Slovenly,  Defective,  Stingy,  Reveal,  Diligent,  Join,  Im¬ 
poverish,  Permanent,  Elation,  Sinful,  Obnoxious,  Conservative, 
Victorious,  Obscure,  Proficient,  Rigid,  Repulsion,  Imaginary, 
Permit,  Orthodoxy,  Analytical,  Extrinsic,  Sacred,  Dynamic,  Lo¬ 
quacious,  Heterogeneous,  Spurious,  Disastrous,  Facility,  Pride, 
Result,  Adroit,  Parsimony,  Suave,  Esoteric. 

One  hundred  and  twelve  subjects  were  tested  with  this  list. 
This  means  that  later  in  determining  the  relative  difficulty  of 
the  stimuli,  the  calculations  are  based  on  at  least  one  hundred 
and  twelve  records  for  each  word,  varying  up  to  one  hundred 
and  eighty-seven  for  others. 

1  This  research  was  conducted  under  the  supervision  of  Dr.  Edward  K. 
Strong  up  to  this  point,  thereafter  under  the  guidance  of  Dr.  Joseph 
Peterson. 


V. 


The  Choice  of  the  Acceptable  Responses 

The  next  step  was  to  standardize  and  evaluate  the  responses. 

In  order  to  facilitate  matters,  beneath  each  stimulus  was  written 
the  long  list  of  words  which  had  been  given  as  opposites  by  those 
subjects  experimented  upon.  The  credit  due  each  of  these  re¬ 
sponses  was  determined  by  five  judges,  including  the  writer. 

They  were  besides  the  writer : 

Miss  Lula  O.  Andrews,  Professor  of  English;  Miss  Mary  Clay 
Hiner,  Instructor  in  English;  Mr.  S.  H.  Phelps,  Instructor  in 
School  Administration,  and  Dr.  Joseph  Peterson,  Professor  of 
Psychology,  all  of  George  Peabody  College. 

Each  judge  was  ignorant  of  the  credit  assigned  any  word  by 
any  other  judge.  The  following  directions,  a  copy  of  which  was 
handed  each  judge,  will  make  the  matter  clear: 

Directions  for  Grading  Responses 

I.  Make  use  of  any  available  source  of  information  such  as  < 

the  dictionary  or  book  of  antonyms. 

II.  The  responses  are  to  be  graded  with  a  grade  of  “one,” 
which  means  an  exact  opposite,  or  “one-half,”  which  means 
only  an  approximate  opposite,  or  “zero,”  which  means  a  failure. 

III.  More  than  one  response  to  a  word  may  be  given  a  credit 
of  “one.” 

IV.  Words  belonging  to  a  different  part  of  speech  are  to  be 
graded  “zero,”  also  words  with  the  prefix  “non.” 

V.  If  a  word  belongs  to  the  same  part  of  speech  and  is  nearly 
an  opposite,  give  it  “one-half”  credit. 

VI.  Add  any  response  that  may  suggest  itself  to  you  and 
grade  it  as  directed  above. 

The  new  words  suggested  were  in  turn  passed  on  by  the  other 
four  judges.  In  addition,  when  later  the  test  was  converted 
into  a  group  test  and  hundreds  of  records  secured,  a  number  of 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


25 


different  responses,  given  by  those  tested,  were  submitted  to  the 
same  process  of  evaluation. 

The  credits  assigned  each  response  were  then  averaged  and 
if  the  result  was  nearer  “zero”  than  “one-half,”  the  word  re¬ 
ceived  no  credit;  if  nearer  “one-half”  than  “zero”  or  “one”  it 
received  “one-half”  credit;  and  if  nearer  “one”  than  “one-half,” 
it  was  given  a  credit  of  “one.”'  For  instance  the  response 
“young”  in  answer  to  “ancient'’  received  credits  as  follows: 


TABLE  IX 


I 

II 

III 

/ 

IV 

Credit 

V  Total  Aver.  Assigned 

Credits  given 

to  “young” 

by  five  j  udges . .  1 

0 

0 

y2  2  .4  y2 

On  the  following  pages  is  to  be  found  in  Column  II  the  stim¬ 
ulus  word,  in  Column  III  the  responses  which  are  due  a  credit  of 
“one’  (the  full  value  assigned  to  the  word),  and  in  Column  IV 
those  responses  which  receive  only  “one-half”  credit  (half  the 
value  assigned  the  word).  The  value  of  the  word  is  given  in 
points  in  Column  I.  The  manner  in  which  we  arrived  at  these 
values  will  be  discussed  in  the  following  section. 


TABLE  X 

A  List  of  the  Stimuli  with  Assigned  Values  and  Accepted  Responses 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

1 

Full 

Empty 

Meagre 

1 

Negative 

Affirmative 

Positive 

Assertive 

1 

After 

Before 

Preceding 

Fore 

I 

Dim 

Bright  ' 

Light 

Clear 

Plain 

Distinct 

Luminous 

Vivid 

1 

Blunt 

Keen 

Acute 

Pointed 

Polite 

Sharp 

Sensitive 

Tactful 

Suave 

26 


MARIE  HACKL  MEANS 


TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

i 

Success 

Defeat 

Downfall 

Disaster 

Failure 

Loss 

i 

Pessimistic 

Optimistic 

Joyful 

I 

Joy 

Gloom 

Depression 

Grief 

Despondency 

Misery 

Displeasure 

Sadness 

Pain 

Sorrow 

Woe 

2 

Public 

Private 

Domestic 

Personal 

Secluded 

Secret 

2 

Profit 

Loss 

Deficit 

Lose 

2 

Spend 

Earn 

Accumulate 

Hoard 

Get 

Husband 

Hold 

Keep 

Make 

Save 

Receive 

2 

Always 

Never 

Infrequently 

Seldom 

2 

Graceful 

Awkward 

Uncouth 

Clumsy 

Ungraceful 

Gawky 

Ungainly 

Unsightly 

2 

Strength 

Feebleness 

Delicacy 

Frailty 

Weakness 

Insecurity 

3 

Ancient 

Modern 

New 

Present 

Recent 

Young 

3 

Expand 

Contract 

Compress 

Decrease 

Diminish 

Shrink 

Shrivel 

Narrow 

3 

Barbarous 

Civilized 

Chivalrous 

Humane 

Civil 

Cultured 

Educated 

Gentle 

Kind 

Polite 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


27 


TABLE  X— Continued 


Value  Stimulus  Word  Correct  Responses  Half  Correct  Responses 


3  Hinder 


3  Despondent 


3  Vague 


3  Fertile 


4  Doubtful 


Aid 

Assist 

Expedite 

Fore 

Forward 

Further 

Facilitate 

Help 

Promote 

Buoyant 

Cheerful 

Ecstatic 

Elated 

Exuberant 

Glad 

Happy 

Hopeful 

Joyful 

Jubilant 

Clear 

Definite 

Distinct 

Exact 

Plain 

Specific 

Arid 

Barren 

Poor 

Sterile 

Unproductive 

Assured 

Certain 

Evident 

Hopeful 

Sure 


4  Injurious  Advantageous 

Beneficial 

Helpful 

Innocuous 

Wholesome 

4  Busy  Idle 

Unemployed 

Unoccupied 


Advance 

Encourage 


Bright 

Encouraged 

Exhilarated 

Gay 

Hilarious 

Joyous 

Merry 

Optimistic 

Sanguine 


Apparent 

Concise 

Explicit 


Fruitless 

Impotent 

Impoverished 

Infertile 

Unimaginative 

Apparent 

Believable 

Clear 

Confident 

Credible 

Known 

Positive 

True 

Truthful 

Unquestioned 

Harmless 

Healthful 

Safe 

Uninjurious 


Dull  _ 
Inactive 
Indolent 
Loafing 


28 


MARIE  HACKL  MEANS 


TABLE  X — Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

4 

Abstract 

Concrete 

Specific 

4 

Advance 

Decline 

Degrade 

Recede 

Hinder 

Retard 

Retire 

Retreat 

Retrograde 

Withdraw 

Withhold 

4 

Foreign 

Domestic 

American 

Germane 

Home 

Native 

Indigenous 

Pertinent 

Local 

Natural 

4 

Create 

Annihilate 

Abolish 

Demolish 

Disintegrate 

Destroy 

Dismember 

Exterminate 

Obliterate 

Undo 

Waste 

4 

Simple 

Complex 

Bright 

Complicated 

Confusing 

Compound 

Difficult 

Elaborate 

Gorgeous 

Intricate 

Grand 

Ornate 

Hard 

Wise 

Learned 

Sensible 

Smart 

4 

Extravagant 

Economical 

Careful 

Frugal 

Close 

Miserly 

Conservative 

Niggardly 

Economizing 

Parsimonious 

Penurious 

Saving 

Stingy 

Thrifty 

4 

Aristocratic 

Bourgeois 

Humble 

Common 

Low 

Democratic 

Lowly 

Plebeian 

Ordinary 

4 

Rare 

Abundant 

Commonplace 

Common 

Often 

Dense 

Ordinary 

Frequent 

Profuse 

Numerous 

Regular 

Plentiful 

Plenteous 

Usual 

Occasional 

4 

Dangerous 

Harmless 

Safe 

Peaceful 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


29 


TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

5 

Slovenly 

Neat 

Careful 

Neatly 

Carefully 

Tidy 

Gracefully 

Tidily 

Precise 

Trim 

5 

Defective 

Complete 

Correct 

Faultless 

Effective 

Normal 

Good 

Perfect 

Sound 

Whole 

Healthy 

5 

Stingy 

Generous 

Bounteous 

Lavish 

Bountiful 

Liberal 

Extravagant 

Prodigal 

Freehearted 

Magnanimous 

Unselfish 

Unsharing 

Wasteful 

5 

Reveal 

Conceal 

Cover 

Hide 

Secrete 

Obscure 

Suppress 

Withhold 

5 

Diligent 

Dilatory 

Careless 

Indolent 

Negligent 

Lazy 

Inattentive 

i 

Slothful 

Idle 

Shiftless 

5 

Join 

Abandon 

Detach 

Disjoin 

Disassociate 

Disconnect 

Disband 

Dismember 

Leave 

Divide 

Resign 

Part 

Tear 

Separate 

Undo 

Sever 

Untie 

Sunder 

Withdraw 

6 

Impoverish 

Enrich 

Aggrandize 

Replenish 

Fertilize 

Nourish 

Strengthen 

7 

Permanent 

Ephemeral 

Ephemerate 

Evanescent 

Fluctuating 

Fleeting 

Impermanent 

Passing 

Perishable 

Temporary 

Shifting 

Transient 

Transitional 

Transitory 

Unsubstantial 

30 


MARIE  HACKL  MEANS 


TABLE  X — Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

7 

Elation 

Dejection 

Depression 

Despair 

Despondence 

Despondency 

Grief 

Sadness 

Shame 

Sorrow 

7 

Sinful 

Blameless 

Righteous 

Sinless 

Godly 

Holy 

Innocent 

Just 

Religious 

Perfect 

Pious 

Pure 

Upright 

Virtuous 

7 

Obnoxious 

Agreeable 

Beneficial 

Inoffensive 

Pleasing 

Acceptable 

Attractive 

Congenial 

Desirable 

Harmless 

Helpful 

Likable 

Pleasant 

Winning 

7 

Conservative 

Extreme 

Liberal 

Progressive 

Radical 

Aggressive 

Extravagant 

Extremist 

Wasteful 

7 

Victorious 

Conquered 

Defeated 

Vanquished 

Beaten 

Unsuccessful 

Unvictorious 

Whipped 

7 

Obscure 

Clear 

Conspicuous 

Eminent 

Lucid 

Plain 

Prominent 

Renowned 

Reveal 

Apparent 

Bright 

Disclose 

Distinct 

Evident 

Famous 

Noticeable 

Notorious 

Obvious 

Unambiguous 

8 

Proficient 

Deficient 

Inapt 

Unskilled 

Unskillful 

Backward 

Incapable 

Incompetent 

Inefficient 

Lacking 

Unprepared 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


3i 


TABLE  X — Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

8 

Rigid 

Elastic 

Easy 

Flexible 

Flabby 

Lax 

Flaccid 

Lenient 

Loose 

Limber 

Limp 

Plastic 

Relaxed 

Pliable 

Supple 

Yielding 

8 

Repulsion 

Attraction 

Acceptance 

Cohesion 

Agreeableness 

Admiration 

Love 

9 

Imaginary 

Real 

Actual 

Prosaic 

Realistic 

True 

9 

Permit 

Forbid 

Deny 

Prevent 

Disallow 

Refuse 

Disapprove 

Restrain 

Object 

Prohibit 

9 

Orthodoxy 

Heresy 

Catholicity 

Heterodoxy 

Liberalism 

Unconventionality 

Unsoundness 

10 

Analytical 

Synthetic 

Synthetical 

Unanalytical 

10 

Extrinsic 

Intrinsic 

Essential 

Internal 

11 

Sacred 

Defiled 

Common 

Profane 

Desecrated 

Secular 

Sacrilegious 

Unholy 

Unconsecrated 

Ungodly 

• 

Vulgar 

11 

Dynamic 

Inert 

Potential 

Static 

Powerless 

Weak 

11 

Loquacious 

Laconic 

Mute 

Reticent 

Reserved 

Taciturn 

Quiet 

Silent 

Untalkative 

12 

Heterogeneous 

Homogeneous 

Alike 

Similar 


32 


MARIE  HACKL  MEANS 


TABLE  X — Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

12 

Spurious 

Authentic 

Genuine 

Real 

* 

Actual 

Legitimate 

Pure 

Sound 

True 

Truthful 

Unadulterated 

13 

Disastrous 

Advantageous 

Beneficial 

Helpful 

Safe 

Favorable 

Fortunate 

Gainful 

Harmless 

Lucky 

Prosperous 

14 

Facility 

t 

k 

Difficulty 

Awkwardness 

Clumsiness 

Disability 

Effort 

Inconvenience 

14 

Pride 

Humility 

Lowliness 

Meekness 

Debasement 

Degradation 

Humbleness 

Humiliation 

15 

Result 

Causation 

Cause 

Beginning 

Commencement 

Origin 

Purpose 

Start 

15 

Adroit 

Awkward 

Crude 

Clumsy 

Unskilled 

1 7 


Parsimony 


22 


Suave 


Inexpert 

Maladroit 

Unskillful 

Extravagance 

Generosity 

Lavishness 

Liberality 

Prodigality 

Brusque 

Impolite 

Tactless 

Unpolished 


Freeheartedness 

Magnanimity 

Wastefulness 


Abrupt 

Blunt 

Crude 

Gruff 

Harsh 

Rough 

Rude 

Uncouth 

Undiplomatic 

Unpleasant 

Untactful 


Esoteric 


Private 


26 


Exoteric 


VI. 


The  Computation  of  the  Relative  Difficulty 

of  the  Stimuli 

As  stated  previously,  when  the  test  was  administered,  each 
response  given  by  the  subject  as  well  as  the  reaction  time  of 
that  response  was  recorded.  On  the  basis  of  these  records  the 
following  data  were  tabulated  for  each  stimulus:  the  different 
responses,  the  reaction  time  of  each  response,  and  the  failures 
to  respond.  Records  were  so  kept  that  information  could  be 
obtained  not  only  regarding  the  number  of  subjects  giving  a 
certain  response  but  also  regarding  the  reaction  times  of  that 
response.  For  instance,  instead  of  merely  indicating  the  number 
of  people  who  gave  “young”  as  an  opposite  to  “ancient,”  the 
records  show  that  “young”  was  given  one  time  in  six-fifths  of 
a  second,  another  time  in  nine-fifths,  another  in  seven,  etc. 

Furthermore,  in  order  to  ascertain  when  a  sufficiently  large 
number  of  subjects  had  been  tested  for  the  results  to  be  reliable 
and  to  be  able  to  indicate  the  extent  of  this  reliability,  as  deter¬ 
mined  by  the  change  brought  about  by  additional  sampling,  sepa¬ 
rate  records  were  kept  of  all  tests  made  prior  to  the  fall  of  1918. 
These  will  be  designated  as  Group  I.  Group  II  consists  of  the 
fifty-six  records  secured  in  the  early  fall  of  1918,  and  Group 
III  of  the  remaining  fifty-six  records.  Since  some  of  the  stimuli 
were  added  to  the  list  at  different  times  the  number  of  records  of 
subjects  in  Group  I  varies  from  zero  to  seventy-nine. 

After  the  acceptable  responses  had  been  determined  these 
record  sheets  were  scored  and  the  number  of  correct  responses 
the  number  of  half  correct  responses,  and  the  number  of  failures, 
as  well  as  the  total  of  all  three,  were  tabulated.  The  number  of 
failures  includes  both  the  failures  to  give  any  opposite  and  the 
failures  to  give  the  correct  opposite.  These  results  are  to  be 
found  in  Table  XI,  columns  III,  IV,  V  and  VI.  Referring  to 
this  table,  we  read  that  for  the  word  “full,”  there  were  in  Group 


34 


MARIE  HACKL  MEANS 


i,  sixty  correct  responses,  no  half  correct  responses,  and  three 
failures,  making  a  total  of  sixty-three  records  for  that  word  ob¬ 
tained  before  the  fall  of  1918.  Reading  further  in  Group  II  for 
the  same  word,  we  have  fifty-four  correct  and  one  half  coirect 
responses  with  one  failure,  thus  totaling  fifty-six.  In  Group  III 
we  find  the  following  distribution:  fifty-five  correct  and  one  half- 
correct. 

In  computing  the  percentage  of  failures,  to  be  found  in  column 
VII,  two  half-correct  responses  were  considered  equivalent  to  one 
failure.  The  formula  will  make  clear  the  procedure  followed. 

H 

F  +  T 

Percentage  of  failures  =  1 -  in  which 

*  N 

F  =  the  number  of  failures, 

H  =  the  number  of  half -correct  responses, 

N  =  the  total  number  tested. 

Applying  this  formula  to  the  word  “full”  we  have  in  Group  I, 

o 

3  H - 

2 

- y - =  -047 

63 

Group  II  does  not  represent  the  percentage  of  failures  in  Group 

II  alone  but  a  combination  of  Groups  I  and  II.  Applying  the 

4  “t”  -5 

formula  again  for  the  same  word  we  have  - =  .037 

119 

in  which  4  equals  the  sum  of  the  failures  in  groups  I  and  II, 
.5  equals  half  of  the  half-correct  responses  in  groups  I  and  II, 
and  1 19  is  the  sum  of  the  total  number  tested.  Similarly  Group 

III  represents  the  sum  of  groups  I,  II,  and  III. 

This  procedure  of  estimating  the  three  different  percentages 
of  failures,  instead  of  only  one  was  adopted  because  it  indicated 
the  degree  of  reliability  of  the  measure.  Very  radical  changes 
obviously  would  mean  little  reliability.  Efforts  were  made  to 
have  our  sampling  as  representative  of  all  the  students  as  possible. 
Therefore  the  number  tested  include  graduate  students  as  well  as 
members  of  the  four  different  classes  of  undergraduates. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


35 


TABLE  XI 

Data  from  Which  the  Percentage  of  Failures  for  Each 

Stimulus  was  Computed 


I 

Stimulus 

II 

Group 

hi 

Number  of 
Correct 
Responses 

IV 

Number  of 
Half-Correct 
Responses 

V 

Number 

of 

Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Full 

I 

60 

0 

3 

63 

.047 

II 

54 

I 

I 

56 

•037 

III 

55 

I 

0 

56 

.028 

Negative 

I 

48 

0 

2 

50 

.040 

II 

53 

0 

3 

56 

.047 

III 

56 

0 

0 

56 

.030 

After 

I 

7 1 

0 

3 

74 

.040 

II 

55 

0 

1 

5b 

.030 

III 

53 

0 

3 

56 

•037 

Dim 

I 

66 

8 

1 

75 

.066 

II 

50 

3 

3 

56 

.072 

III 

42 

14 

0 

56 

.089 

Blunt 

i 

60 

0 

3 

63 

•947 

n 

48 

2 

6 

56 

‘.084 

hi 

50 

0 

6 

56 

.091 

Success 

i 

59 

0 

4 

63 

.063 

ii 

5i 

1 

4 

56 

.071 

hi 

44 

3 

9 

56 

.108 

Pessimistic 

i 

47 

0 

2 

49 

.040 

ii 

49 

1 

6 

56 

.080 

hi 

47 

0 

9 

56 

.108 

Joy 

i 

65 

0 

3 

68 

.044 

ii 

49 

3 

4 

56 

.068 

in 

48 

2 

6 

56 

.086 

Public 

i 

63 

0 

5 

68 

•073 

ii 

46 

3 

7 

56 

.108 

in 

49 

3 

4 

56 

.105 

Profit 

i 

55 

7 

5 

67 

.126 

ii 

52 

0 

4 

56 

.101 

hi 

4i 

5 

10 

56 

•139 

Spend 

i 

46 

9 

8 

63 

.198 

ii 

5i 

1 

4 

56 

.142 

in 

5i 

I 

4 

56 

.122 

Always 

i 

52 

1 

10 

63 

.166 

ii 

44 

0 

12 

56 

.188 

hi 

43 

2 

11 

56 

.197 

36 


MARIE  HACKL  MEANS 


TABLE  XI — Continued 


I 

Stimulus 

II 

Group 

in 

Number  of 
Correct 
Responses 

IV 

Number  of 
Half-Correct 
Responses 

V 

Number 

of 

Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Graceful 

I 

50 

13 

5 

68 

.169 

II 

44 

7 

5 

56 

.161 

III 

45 

4 

7 

56 

.161 

Strength 

I 

56 

0 

8 

64 

•125 

II 

43 

0 

13 

56 

•  175 

III 

4i 

0 

15 

56 

.204 

Ancient 

I 

33 

26 

3 

62 

.258 

II 

32 

22 

2 

56 

•245 

III 

40 

14 

2 

56 

.218 

Expand 

I 

45 

1 

11 

57 

.201 

II 

43 

2 

11 

56 

.207 

III 

41 

0 

15 

56 

.227 

Barbarous 

I 

57 

11 

0 

68 

.080 

II 

34 

17 

5 

56 

.153 

III 

34 

12 

10 

56 

.194 

Hinder 

I 

52 

3 

13 

68 

.213 

II 

44 

0 

12 

56 

.213 

III 

39 

5 

12 

56 

.227 

Despondent 

I 

48 

12 

3 

63 

.142 

II 

37 

15 

4 

56 

.172 

III 

39 

8 

9 

56 

.191 

Vague 

I 

55 

2 

10 

67 

.164 

II 

42 

2 

12 

56 

•195 

III 

38 

1 

1 7 

56 

.231 

Fertile 

I 

54 

1 

15 

7o 

.221 

n 

40 

0 

16 

56 

.250 

in 

47 

0 

9 

56 

.222 

Doubtful 

i 

38 

9 

10 

57 

•254 

1 1 

39 

9 

8 

56 

.238 

hi 

38 

10 

8 

56 

.236 

Injurious 

i 

48 

10 

10 

68 

.220 

ii 

39 

8 

9 

56 

.225 

hi 

36 

6 

14 

56 

•249 

Busy 

i 

39 

4 

24 

67 

.388 

ii 

35 

1 

20 

56 

.378 

hi 

4i 

1 

14 

56 

•340 

Abstract 

i 

0 

0 

0 

0 

.000 

ii 

35 

2 

19 

56 

•357 

in 

33 

4 

19 

56 

•365 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


37 


TABLE  XI — Continued 


I 

Stimulus 

II 

Group 

III 

Number  of 
Correct 
Responses 

IV 

Number  of 
Half-Correct 
Responses 

V 

Number 

of 

Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Advance 

I 

48 

0 

10 

58 

.172 

II 

32 

4 

20 

56 

.280 

III 

34 

1 

21 

56 

•314 

Foreign 

I 

37 

TO 

6 

53 

.207 

11 

34 

10 

12 

56 

.256 

III 

34 

10 

12 

56 

.272 

Create 

I 

50 

3 

14 

67 

.231 

11 

38 

1 

1 7 

56 

.292 

III 

35 

2 

19 

56 

.296 

Simple 

I 

13 

21 

9 

43 

•453 

II 

34 

15 

7 

56 

•343 

III 

32 

14 

10 

56 

.328 

Extravagant 

I 

4i 

8 

7 

56 

.196 

II 

42 

6 

8 

56 

.196 

III 

35 

9 

12 

56 

.229 

Aristocratic 

I 

52 

4 

12 

68 

.205 

II 

34 

5 

17 

56 

.270 

III 

32 

1 

23 

56 

.316 

Rare 

I 

37 

16 

8 

61 

.262 

II 

38 

7 

11 

56 

.260 

III 

38 

„6 

12 

56 

.262 

Dangerous 

I 

35 

1 

15 

5i 

.303 

II 

36 

0 

20 

56 

•331 

III 

35 

0 

21 

56 

•346 

Slovenly 

I 

44 

10 

14 

68 

.279 

II 

27 

11 

18 

56 

•342 

III 

28 

4 

24 

56 

.380 

Defective 

I 

47 

6 

15 

68 

.264 

II 

38 

6 

12 

56 

.264 

III 

27 

8 

21 

56 

322 

Stingy 

I 

43 

11 

14 

68 

.286 

II 

30 

11 

15 

56 

.322 

III 

18 

13 

25 

56 

•397 

Reveal 

I 

38 

4 

21 

63 

.365 

II 

31 

4 

20 

56 

.378 

III 

34 

4 

18 

56 

•371 

Diligent 

I 

43 

4 

20 

67 

.328 

II 

33 

15 

8 

56 

.304 

III 

26 

9 

21 

56 

•351 

38 


MARIE  HACKL  MEANS 


i 

TABLE  XI — Continued 


I 

Stimulus 

II 

Group 

hi 

Number  of 
Correct 
Responses 

IV 

Number  of 
Half-Correct 
Responses 

V 

Number 

of 

Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Join 

I 

33 

11 

11 

55 

.300 

II 

38 

0 

18 

56 

.310 

III 

30 

7 

19 

56 

•341 

Impoverish 

I 

38 

4 

27 

69 

.420 

n 

22 

3 

3i 

56 

.492 

hi 

27 

0 

29 

56 

.500 

Permanent 

i 

37 

5 

21 

63 

•373 

n 

24 

1 

3i 

56 

.462 

hi 

34 

1 

21 

56 

•437 

Elation 

i 

33 

10 

19 

62 

•387 

ii 

1 7 

3 

36 

56 

.521 

hi 

18 

3 

35 

56 

•563 

Sinful 

i 

20 

40 

8 

68 

.411 

ii 

13 

25 

18 

56 

.471 

hi 

1 7 

19 

20 

56 

.488 

Obnoxious 

i 

43 

11 

13 

67 

.276 

ii 

14 

22 

20 

56 

.402 

hi 

16 

14 

26 

56 

.460 

Conservative 

i 

39 

3 

27 

69 

•413 

ii 

15 

15 

26 

56 

.496 

hi 

14 

15 

27 

56 

•533 

Victorious 

i 

43 

4 

24 

67 

.388 

ii 

22 

9 

25 

56 

•451 

hi 

24 

8 

24 

56 

.466 

Obscure 

i 

27 

4 

26 

57 

.491 

ii 

25 

3 

28 

56 

.508 

hi 

25 

6 

25 

56 

.505 

Proficient 

i 

18 

30 

15 

63 

.476 

ii 

16 

18 

22 

56 

.512 

hi 

14 

13 

29 

56 

•551 

Rigid 

i 

27 

14 

16 

57 

403 

ii 

1 7 

18 

21 

56 

.469 

hi 

14 

18 

24 

56 

.508 

Repulsion 

i 

29 

4 

34 

67 

•537 

ii 

20 

2 

34 

56 

•5  77 

hi 

18 

3 

35 

56 

.600 

Imaginary 

i 

22 

4 

28 

54 

•555 

ii 

21 

2 

33 

5b 

.581 

hi 

20 

2 

34 

56 

•596 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


39 


TABLE  XI— Continued 


I 

Stimulus 

II 

Group 

III 

Number  of 
Correct 
Responses 

IV 

Number  of 
Half-Correct 
Responses 

V 

Number 

of 

Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Permit 

I 

28 

12 

28 

68 

.500 

II 

24 

7 

25 

56 

004 

III 

24 

8 

24 

56 

.502 

Orthodoxy 

I 

15 

0 

40 

55 

.727 

II 

9 

0 

47 

56 

•783 

III 

6 

0 

50 

56 

.820 

Analytical 

I 

29 

1 

27 

57 

.482 

II 

10 

1 

45 

56 

.646 

III 

11 

0 

45 

56 

.698 

Extrinsic 

I 

1 

0 

1 

2 

.500 

II 

16 

0 

40 

56 

.706 

III 

15 

0 

4i 

56 

.719 

Sacred 

I 

24 

4 

30 

58 

•551 

II 

15 

7 

34 

56 

.609 

III 

10 

11 

35 

56 

.647 

Dynamic 

I 

5 

4 

30 

39 

.820 

II 

3 

4 

49 

56 

•873 

III 

7 

2 

47 

56 

.867 

Loquacious 

I 

14 

20 

28 

62 

.612 

II 

13 

3 

40 

56 

•673 

III 

13 

5 

38 

56 

.689 

Heterogeneous 

I 

2 

0 

1 

3 

•333 

II 

9 

4 

43 

56 

•779 

III 

12 

0 

44 

56 

.782 

Spurious 

I 

6 

12 

3i 

49 

•755 

i 

II 

4 

3 

49 

56 

.833 

III 

4 

3 

49 

56 

.857 

Disastrous 

I 

18 

2 

30 

50 

.620 

II 

15 

6 

35 

56 

.650 

III 

21 

2 

33 

56 

.635 

Facility 

I 

10 

1 

34 

45 

.766 

II 

6 

3 

47 

56 

.821 

III 

5 

0 

51 

56 

•853 

Pride 

I 

17 

3 

53 

73 

.746 

II 

8 

2 

46 

56 

.786 

III 

6 

1 

49 

56 

.816 

Result 

I 

15 

15 

32 

62 

.636 

II 

10 

6 

40 

56 

.698 

III 

4 

6 

46 

56 

•755 

40 


MARIE  HACKL  MEANS 


TABLE  XI — Continued 


I 

Stimulus 

II 

Group 

III 

Number  of 
Correct 
Responses 

IV  V 

Number  of  Number 
Half-Correct  of 
Responses  Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Adroit 

I 

19 

2 

40 

61 

.672 

II 

7 

3 

46 

56 

756 

III 

3 

0 

53 

56 

.81 7 

Parsimony 

I 

11 

2 

35 

48 

•750 

II 

7 

0 

49 

56 

.816 

III 

6 

6 

44 

56 

.823 

Suave 

I 

2 

10 

34 

46 

.847 

II 

2 

9 

45 

56 

.867 

III 

1 

8 

47 

56 

.882 

Exoteric 

I 

3 

0 

49 

52 

.942 

II 

1 

0 

55 

56 

.962 

III 

0 

0 

56 

56 

•975 

Since  the  value  of  the  stimuli  was  to  be  in  terms  of  both  ac¬ 
curacy  and  speed,  the  median  time  of  all  responses,  both  correct 
and  half  correct,  was  computed.  The  time  score  taken  with  a 
stop-watch  represents  fifths  of  a  second.  As  in  the  case  of  the 
accuracy  score,  three  time  scores  were  reckoned  for  each  word, 
the  last  score  representing  the  combined  time  of  all  three  groups. 
See  column  III,  Table  XII.  The  percent  of  failures  of  the  pre¬ 
ceding  table  is  copied  in  order  to  make  clear  the  process  by  which 
the  values  of  the  several  stimuli  are  reached. 

To  arrive  at  the  relative  difficulty  of  the  stimuli,  the  percen¬ 
tage  of  failures  was  multiplied  by  the  median  time.  If  credit  is 
to  be  given  for  both  measures,  some  more  or  less  arbitrary 
scheme  must  be  adopted.  This  procedure  seemed  justifiable  since 
by  it  words  of  equal  accuracy  scores  received  different  point 
values  if  the  difference  in  time  varied  to  any  appreciable  extent. 
This  was  precisely  what  was  desired.  Any  method  which  covered 
up  this  difference  was  to  be  avoided. 

In  order  to  determine  a  point  value  for  the  stimuli,  the  stan¬ 
dard  deviation  of  the  different  values,  as  indicated  in  the  fourth 
column  of  the  following  table,  was  obtained.  Beginning  at  the 
average  and  calling  it  zero,  steps  of  one-fifth  standard  deviation 
were  then  made,  positively  and  negatively.  These  steps  being 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


4i 


equal  in  a  sense,  the  furtherest  negative  step  was  assigned  a  value 
of  one,  the  next  two,  and  so  on  up  to  twenty-six,  the  most  difficult 
word  in  the  list.  These  point  values  are  given  in  the  fifth  column 
of  Table  XII. 

In  selecting  the  stimuli  for  the  completed  test,  those  words 
which  had  received  a  hundred  percent  accuracy  score  were  dis¬ 
carded,  for  thus  we  were  better  enabled  to  locate  a  zero  point. 
Obviously  we  cannot  find  or  infer  the  place  where  the  ability 
to  give  opposites  begins,  as  we  can  in  the  physical  sciences  for 
weight  or  length.  Therefore  the  zero  point  will  of  necessity  be 
an  arbitrary  one  selected  to  represent  very  little  of  the  trait  in 
question.  In  eliminating  all  words  with  a  hundred  percent  ac¬ 
curacy  score,  but  retaining  those  next  in  difficulty,  we  assume 
that  a  college  student  who  is  unable  to  score  a  point  on  the  test 
as  it  now  stands  has  zero  ability  in  naming  opposites.  Since  we 
have  located  the  distance  from  the  median  in  terms  of  standard 
deviation,  we  have  equally  difficult  steps.  Roughly  we  assigned 
to  the  furtherest  negative  step  a  value  of  one.  Since  we  are  not 
concerned  as  were  Woody1  and  Hering2  with  finding  the  dis¬ 
tances  between  different  grades  or  groups  of  pupils,  it  seems 
impracticable  to  be  too  particular  about  the  absolute  point  of 
the  zero.  In  addition  any  error  in  slightly  misplacing  the  point 
might  be  of  less  importance  than  that  made  in  weighting  time 
and  accuracy. 

It  is  regretted  that,  especially  in  the  case  of  the  more  difficult 
words,  the  measure  of  time  is  based  on  so  small  a  number  of 
records.  As  an  indication  of  the  reliability  of  the  median  time,  we 
have  computed  the  median  deviation  and  coefficient  of  variability 
for  each  word.  The  probable  error  of  the  median  is  calculated 

5  Q  dis 

by  the  formula,  P.  E.  med.  =  - - .3 

4Vn 

1  Woody,  Clifford,  “Measurements  of  Some  Achievements  in  Arithmetic,” 
Col.  Univ.  Cont.  Educ.,  1916,  LXXX. 

2  Hering,  John  P.,  “Derivation  of  a  Scale  to  Measure  Abilities  in  Scientific 
Thinking,”  Jour.  Educ.  Psychol.,  1919,  IX,  417-431. 

3  Thorndike,  E.  L.,  “Mental  and  Social  Measurements,”  New  York,  1916, 
p.  195- 


42 


MARIE  HACKL  MEANS 


TABLE  XII 

The  Point  Value  of  the  Stimuli  Computed  from  Table  IX 

VII 


I 

Stimulus 

II 

Percent 

of 

Failures 

III  IV 

Median  Relative 
Time  Value 

V 

Point 
Value  ' 

VI 

Mediar 

Deviatic 

Full 

.047 

7 

.329 

•037 

6 

.222 

.028 

7 

.209 

1 

1.48 

Negative 

.040 

7 

.280 

.047 

7 

•329 

.030 

7 

.210 

1 

1.22 

After 

.040 

6 

.240 

.030 

7 

.210 

•037 

7 

.259  . 

1 

1.20 

Dim 

.066 

8 

.528 

.072 

7 

•504 

.089 

7 

.623 

1 

1.64 

Blunt 

.047 

6 

.282 

.084 

7 

.588 

.091 

7 

.637 

1 

1.40 

Success 

.063 

6 

.378 

.071 

7 

•497 

.108 

6 

.648 

1 

1.50 

Pessimistic 

.040 

6 

.240 

.080 

6 

.480 

.108 

6 

.648 

1 

1. 11 

Joy 

.044 

7 

.308 

. 

.068 

8 

•544 

.086 

8 

.688 

1 

10 

b 

CO 

Public 

•073 

7 

.511 

.108 

7 

.756 

4 

.105 

8 

.840 

2 

2.03 

Profit 

.126 

7 

.882 

.101 

8 

.808 

•139 

8 

1.112 

2 

2.26 

Spend 

.198 

10 

1.980 

.142 

10 

1.420 

.122 

10 

1.220 

2 

2.80 

Always 

.166 

8 

1.328 

.188 

8 

1.504 

.197 

8 

1.576 

2 

1.90 

Graceful 

.169 

10 

1.690 

.161 

10 

1.610 

.161 

10 

1.610 

2 

3-36 

VIII 


n  of  Error  of 
Variability  Median 


.21 


•i  7 


.17 


•23 


.20 


•25 


.18 


.26 


•25 


.28 


.28 


.23 


.14 


.12 


.11 


•15 


•13 


.14 


.11 


.20 


.19 


.22 


.27 


.19 


•33 


•32 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


43 


TABLE  XII — Continued 


I 

Stimulus 

II 

Percent 

of 

Failures 

HI  IV 

Median  Relative 
Time  Value 

y  vi  vii  viii 

Point  Median  Coefficient  Probable 
Value  Deviation  of  Error  of 

Variability  Median 

Strength 

•125 

7 

.875 

•  175 

8 

1.400 

.204 

8 

1.632 

2 

1-95 

.24 

.20 

Ancient 

.258 

8 

2.064 

.245 

7-5 

1.837 

.218 

8 

1.744 

3 

2.30 

•25 

.22 

Expand 

.201 

9 

1.809 

•297 

8 

1.656 

.227 

8 

1.816 

3 

1.28 

.16 

•13 

Barbarous 

.080 

11 

.880 

•153 

11 

1.683 

.194 

11 

2.134 

3 

2.90 

.26 

.28 

Hinder 

.213 

9 

1.917 

.213 

10 

2.130 

.227 

10 

2.270 

3 

348 

•34 

.36 

Despondent 

.142 

14 

1.988 

.172 

12 

2.064 

.191 

12 

2.292 

3 

3-54 

•29 

•35 

Vague 

.164 

9 

1.476 

•  195 

10 

1.950 

.231 

11 

2.541 

3 

3.23 

•29 

•34 

Fertile 

.221 

12 

2.652 

.250 

12 

3.000 

.222 

12 

2.664 

3 

3-85 

•32 

.40 

Doubtful 

•254 

12 

3.048 

.238 

12 

2.858 

.236 

12 

2.832 

4 

3-89 

.32 

.40 

Injurious 

.220 

11 

2.420 

.225 

12 

2.700 

.249 

12 

2.988 

4 

447 

•37 

.46 

Busy 

.388 

10 

3.880 

•378 

10 

3.780 

•340 

9 

3.060 

4 

2.64 

•29 

•30 

Abstract 

.000 

0 

0.000 

•357 

9 

3-213 

•365 

8.5 

3.102 

4 

2.10 

•24 

•30 

Advance 

.172 

9 

1.548 

.280 

9 

2.520 

•314 

10 

3.140 

4 

2.90 

•29 

•33 

44 


MARIE  HACKL  MEANS 


TABLE  XII — Continued 


I 

Stimulus 

II 

Percent 

of 

Failures 

HI 

Median 

Time 

IV 

Relative 

Value 

V  VI  VII  VIII 

Point  Median  Coefficient  Probable 
Value  Deviation  of  Error  of 

Variability  Median 

Foreign 

.207 

12 

2.484 

.256 

12 

3.072 

.272 

12 

3.264 

4 

3-8o 

•31 

45 

Create 

.231 

10 

2.310 

.292 

11 

3.212 

.296 

11 

3.256 

4 

3.3i 

•30 

.36 

Simple 

453 

8 

3.624 

•343 

10 

3-430 

.328 

10 

3.280 

4 

3.23 

•32 

.28 

Extravagant 

.196 

14 

2.744 

.196 

15 

2.940 

.229 

15 

3435 

4 

4.14 

•27 

43 

Aristocratic 

.205 

12 

2.460 

.270 

11 

2.970 

•316 

11 

3476 

4 

4.20 

•38 

46 

Rare 

.262 

14 

3.668 

.260 

14 

3-640 

.262 

14 

3.668 

4 

3-39 

.24 

•35 

Dangerous 

•303 

11 

3-333 

.331 

11 

3-641 

.346 

11 

3.806 

4 

3-75 

•34 

45 

Slovenly 

.279 

11 

3-069 

.342 

12 

4.104 

.380 

11 

4.180 

5 

3-62 

.32 

•63 

Defective 

.264 

13 

3432 

.264 

13 

3432 

.322 

13 

4.186 

5 

5-29 

40 

.58 

Stingy 

.286 

10 

2.860 

.322 

12 

3.864 

•397 

11 

4467 

5 

3-34 

•30 

•37 

Reveal 

•365 

11 

4.015 

.378 

11 

4.158 

•371 

12 

4452 

5  f 

3-83 

.31 

•44 

Diligent 

.328 

13 

4.264 

.304 

13 

3-952 

•35i 

13 

4.563 

5 

3-97 

•30 

43 

Join 

.300 

14 

4.200 

.310 

13 

4.030 

•34i 

14 

4-774 

5 

4.90 

•35 

.56 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


45 


TABLE  XII — Continued 


I 

Stimulus 

II 

Percent 

of 

Failures 

HI 

Median 

Time 

IV 

Relative 

Value 

y  vi  vii  viii 

Point  Median  Coefficient  Probable 
Value  Deviation  of  Error  of 

Variability  Median 

Impoverish 

420 

12 

5.040 

.492 

11 

5-412 

.500 

11 

5-500 

6 

•341 

•31 

49 

Permanent 

•373 

13 

4.849 

,462 

13 

6.006 

•437 

14 

6.118 

7 

4.50 

.32 

•55 

Elation 

.387 

10 

3.870 

•521 

12 

6.262 

.563 

12 

6.756 

7 

3-79 

.31 

•51 

Sinful 

.411 

13 

5-343 

.471 

16 

7.536 

.488 

14 

6.832 

7 

5.10 

.36 

.64 

Obnoxious 

.2  76 

13 

3.588 

.402 

14 

5.628 

.460 

15 

6.900 

7 

5.36 

•35 

.61 

Conservative 

•413 

11 

4-543 

.496 

11 

6.448 

•533 

13 

6.929 

7 

4.30 

•33 

•53 

Victorious 

.388 

14 

5-432 

•451 

15 

6.765 

.466 

15 

6.990 

7 

4-75 

.31 

.56 

Obscure 

•491 

12 

5.892 

.508 

13 

6.604 

.505 

14 

7.070 

7 

6.06 

43 

•79 

Proficient 

476 

11 

5.236 

.512 

13 

6.656 

.551 

13 

7.163 

8 

545 

.41 

.65 

Rigid 

403 

15 

6.045 

.469 

15 

7.035 

.508 

15 

7.620 

8 

4-97 

•33 

•59 

Repulsion 

•537 

11 

5.907 

•5  77 

13 

7-501 

.600 

13 

7.800 

8 

5-30 

.40 

•75 

Imaginary 

•555 

11 

6.105 

•581 

12 

6.972 

.596 

14 

8-344 

9 

4.56 

•32 

.67 

Permit 

.500 

16 

8.000 

•504 

16 

8.064 

.502 

17 

8.534 

9 

6.50 

.38 

.80 

46 


MARIE  HACKL  MEANS 


TABLE  XII— Continued 


I 

Stimulus 

II 

Percent 

of 

Failures 

hi  iv 

Median  Relative 
Time  Value 

V  VI  VII  VIII 

Point  Median  Coefficient  Probable 
Value  Deviation  of  Error  of 

Variability  Median 

Orthodoxy 

.727 

11 

7-997 

•783 

11 

8.613 

.820 

11 

9.020 

9 

2.50 

.22 

•57 

Analytical 

.482 

16.5 

7-953 

.646 

14 

9.044 

.698 

14 

9.772 

10 

4.27 

•30 

•74 

Extrinsic 

.500 

9 

4-500 

.706 

17 

12.000 

.719 

14 

10.066 

10 

5.30 

•37 

1.17 

Sacred 

•55i 

15 

8.265 

.609 

16 

9-744 

.647 

16 

10.352 

11 

6.17 

.38 

.91 

Dynamic 

.820 

12 

9.840 

.873 

12 

10.476 

.867 

12 

10.404 

11 

3-75 

•3i 

•93 

Loquacious 

.612 

15 

9.180 

•673 

15 

10.095 

.689 

16 

11.024 

11 

5.08 

•3i 

.76 

Heterogeneou 

s  -333 

14-5 

4.828 

■779 

13 

10.127 

.782 

15 

11.730 

12 

4.88 

•32 

1.17 

Spurious 

•755 

14 

10.570 

.833 

14 

11.662 

•857 

14 

11.998 

12 

5.50 

•39 

1 .21 

Disastrous 

.620 

17 

10.540 

.650 

19 

12.350 

•635 

20 

12.700 

13 

7.08 

•35 

1. 10 

Facility 

.766 

16 

12.256 

.821 

18 

14.778 

•853 

16 

13-648 

14 

7.00 

•43 

1.75 

Pride 

.746 

16 

11.936 

.786 

17 

13.362 

.816 

1 7 

13.872 

14 

5-88 

•34 

1.20 

Result 

.636 

20 

12.740 

.699 

22 

I5.378 

•755 

20 

15.100 

15 

8.00 

.40 

1-33 

Adroit 

.672 

16 

10.752 

•756 

17 

12.852 

.817 

18.5 

I5-II4 

15 

6.23 

•33 

1-33 

STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


47 


TABLE  XII— Continued 


I  II  III  IV  V  VI  VII  VIII 

Percent  Median  Relative  Point  Median  Coefficient  Probable 
Stimulus  of  Time  Value  Value  Deviation  of  Error  of 

Failures  Variability  Median 


Parsimony 

750 

25 

18.750 

.816 

22 

17.952 

.823 

21 

17.183 

1 7 

6.60 

.31 

1-45 

Suave 

•847 

24.5 

20.751 

.867 

25 

21.675 

.882 

25 

22.050 

22 

9.17 

•36 

2.02 

Exoteric 

.942 

36 

33.912 

.962 

27-5 

26.455 

•975 

27.5 

26.812 

26 

9-50 

•34 

5-93 

VII. 


The  Conversion  into  a  Group  Test 

In  order  to  make  it  possible  for  even  the  inexperienced  to  use 
the  test  with  facility,  it  was  converted  into  a  group  test.  Thus, 
obviously,  much  time  in  administering  and  labor  in  scoring  were 
dispensed  with.  In  addition,  the  personal  element  became  almost 
negligible.  The  words  were  arranged  in  the  ascending  order  of 
difficulty  and  six  minutes  allowed  for  the  test.  This  time  limit 
was  determined  empirically,  after  experimenting  upon  a  dozen 
or  more  students  at  the  George  Peabody  College  and  some  sixty 
students  at  the  North  Georgia  Agricultural  College.1  In  arriving  at 
the  time  to  be  allowed,  two  considerations  were  held  in  mind.  It 
was  desired  that  sufficient  time  be  given  for  individual  scores  to 
cover  a  considerable  range,  and  thus  avoid  a  large  undistributed 
group  of  poor  records.  On  the  other  hand  the  time  should  be  so 
planned  that  a  perfect  score  is  impossible.  Two  individuals  with 
a  perfect  score  might  vary  in  efficiency,  but  under  the  conditions 
of  the  test  no  measure  of  their  difference  could  be  obtained. 

The  test  sheets  were  printed  upon  a  good  quality  of  white 
paper,  eight  and  a  half  by  eleven  inches.  The  directions  printed 
on  the  back  of  the  sheet  explained  to  the  experimenter  as  well  as 
to  the  subject,  exactly  what  was  to  be  done.  A  copy  is  included 
in  the  Appendix.  By  mistake  the  word  “rare”  was  placed  be¬ 
tween  “defective”  and  stingy.”  It  should  follow  “aristocratic.” 
In  scoring  the  papers,  however,  proper  credit  was  assigned  the 
word.  The  displacement  was  so  slight  that  it  is  highly  improb¬ 
able  that  the  final  results  were  at  all  affected. 

Three  thousand  copies  of  the  test  were  sent  to  psychologists 
in  twenty  different  normal  schools,  colleges,  and  universities 
throughout  the  country.  Of  this  number,  one  thousand  six  hun¬ 
dred  and  twenty-eight  copies  were  returned  for  grading  from 

1  These  results,  which  were  obtained  through  the  kindness  of  Professor 
George  Camp,  are  not  included  in  the  norms  because  the  sheets  were  mimeo¬ 
graphed  instead  of  printed. 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


49 


fourteen  widely  different  institutions.  Unfortunately,  a  number 
of  papers  arrived  too  late  to  be  included  in  the  norms  herein  pre¬ 
sented.  The  table  below  indicates  the  number  of  subjects  tested 
in  each  school  and  the  class  to  which  both  the  men  and  women 
then  belonged. 

TABLE  XIII 

The  Number  of  Men  and  Women  Tested  in  Each  School  and  Class  in 

Which  They  Were  Enrolled 


Freshmen 

Soph’s 

Juniors 

Seniors 

Graduates 

Total 

Women  Men 

Women  Men 

Women  Men 

Women  Men 

Women  Men 

Wellesley 

College 

43 

95 

138 

Brooklyn 

Training  School 
University 

148 

5 

153 

of  Illinois 
University 

4 

2 

44 

46 

60 

34 

35 

16 

2 

243 

of  Montana 
Alabama 

23 

2 

10 

5 

24 

1 

66 

State  Normal 
University 

50 

50 

of  Chicago 
Teachers’ 

2 

6 

40 

74 

7 

13 

20 

9 

2 

173 

College 

Vanderbilt 

2 

13 

2 

9 

28 

54 

University 

University 

1 

I 

12 

14 

of  Kansas 

1 

32 

27 

11 

6 

3 

4 

I 

85 

University 
of  Tennessee 
University 

I 

25 

19 

7 

2 

4 

1 

2 

60 

of  Louisiana 
Fitchburg 

2 

1 

14 

9 

5 

14 

8 

4 

I 

58 

Normal  School 
Kansas 

54 

1 7 

26 

17 

2 

116 

State  Normal 
Ohio  State 

49 

6 

38 

I 

15 

3 

112 

University 

59 

63 

40 

58 

33 

22 

14 

13 

2 

2 

306 

Total 

319 

101 

375 

237 

262 

102 

121 

5i 

l6 

44 

1628 

The  final  results  of  the  sixteen  hundred  records  indicated  class 
variations  as  well  as  slight  sex  differences.  In  order  to  ascertain 
a  possible  cause,  the  writer  has  computed  the  percentage  of  stu¬ 
dents  of  both  sexes  in  each  of  the  classes.  This  distribution  is 
shown  in  the  following  table. 

In  grading  the  papers  two  classes  of  errors  were  encountered, 
namely,  the  misspelled  word,  and  the  abbreviated  word.  In  the 
former  case  full  credit  was  allowed  whenever  the  spelling  clearly 


50 


MARIE  HACKL  MEANS 


TABLE  XIV 

The  Percentage  of  Students  of  Both  Sexes  in  Each  of  the  Classes 


Freshmen 

Sophomores 

Juniors 

Seniors 

Graduates 

Women 

.291 

•343 

.239 

.110 

.014 

Men 

.188 

•440 

.190 

.095 

.082 

Both  Sexes 

.27 

•37 

.22 

.10 

•03 

showed  that  an  acceptable  word  was  meant.  However,  when  a 
word  similar  in  sound  was  correctly  spelled  no  credit  was  given. 
As  an  example,  “save”  was  not  accepted  as  an  opposite  of  “dan¬ 
gerous/'  but  “richous”  (actual  case)  as  an  opposite  of  “sinful” 
was  given  full  credit.  This  rule  was  adopted  because  without  it 
gradings  by  different  individuals  would  vary  considerably. 

Where  the  response  was  abbreviated  it  was  given  no  credit. 
The  only  two  abbreviations  were  “opt”  as  the  opposite  of  pes¬ 
simistic  and  “dem”  as  opposite  of  aristocratic.  In  both  cases 
the  subject  might  have  intended  to  write  “optimist”  and  “demo¬ 
crat,”  both  of  which  would  have  been  scored  a  failure.  Even 
had  he  had  in  mind  the  adjective  instead  of  the  noun,  his  score 
would  have  been  raised  only  five  points.  No  doubt  in  abbreviat¬ 
ing  the  words,  he  was  thus  enabled,  at  an  unfair  advantage  to 
others,  to  write  the  opposite  of  a  more  difficult  word.  In  view  of 
the  fact  that  a  time  limit  was  placed  on  the  test,  not  to  penalize 
abbreviations  seems  unjust  to  those  students  who  have  followed 
the  directions  carefully.  Fortunately  the  number  of  students 
abbreviating  was  negligible. 

The  sum  of  the  point  values  of  the  stimuli  as  indicated  in  Table 
XII  is  four  hundred  and  thirty-eight.  Since  each  response  may 
receive  the  assigned  value,  one-half  that  value,  or  zero,  almost 
any  score  between  zero  and  this  number  is  possible.  This  wide 
range  of  distribution  eliminates  the  probability  of  a  large  group 
of  subjects  of  different  ability  receiving  identical  scores.  The 
actual  scores  based  on  the  records  obtained  varied  from  zero  to 
three  hundred  and  fifty-one. 


VIII. 

The  Establishment  of  Norms  for  College  Students 

Sixteen  hundred  and  twenty-eight  test  papers  were  graded  and 
the  norms  for  college  students  by  the  group  test  were  based  on 
these  records.  In  order  to  ascertain  the  form  of  the  distribution 
of  the  scores  made  by  the  several  classes,  steps  of  fifty  beginning 
with  zero  were  made  and  the  percent  of  students  included  between 
these  steps  was  estimated.  Examination  of  Table  XV  reveals 
the  fact  that  the  largest  percentage  of  scores  falls  between  one 
hundred  and  one  hundred  fifty.  This  is  perhaps  more  clearly 
shown  in  the  graphs  (Figures  2,  3,  4,  5,  6)  based  on  this  table. 


TABLE  XV 

Percent  of  Students  of  Several  Classes  Attaining  Scores 
Within  the  Indicated  Limits 


Score 

Freshmen 

Sophomores 

Juniors 

Seniors 

Graduates 

0-  49-5 

.069 

•033 

.016 

.005 

.000 

50-  99-5 

.356 

.272 

.190 

t— 1 

00 

.066 

1 00- 1 49.5 

•397 

.346 

.318 

.406 

•300 

I50-I99-5 

.124 

.225 

.247 

.214 

.250 

200-249.5 

•045 

.078 

.156 

.111 

.163 

250-299.5 

.007 

.032 

.063 

.047 

•133 

300-349.5 

.002 

.014 

.010 

.028 

.086 

350-399.5 

.000 

.000 

.000 

.005 

.000 

The  norms  for  college  students  are  presented  in  the  form  of 
percentile  tables  because  this  method,  statistically  simple,  admits 
of  convenient  use  and  permits  comparisons  to  be  made  with 
other  tests,  similarly  treated.  Let  us  suppose  that  a  directions 
test  has  been  standardized  and  the  norms  compiled  in  terms  of 
percentiles.  Scores  made  in  the  opposites  test  can  be  compared 
with  scores  made  in  the  directions  test  when  both  are  converted 
into  percentiles.  The  two  percentiles  can  be  averaged  or  the 
median  computed.  This  is  statistically  impossible  when  the 
two  scores  are  given  in  terms  of  the  score  in  points  made  on 


52 


MARIE  HACKL  MEANS 


o 

•* 


© 

■o 

to 


o 

o 

to 


o 

<o 

c* 


o 

o 

01 


o 

10 


o 

o 


o 

*n 


as 

as 


4>  - 
n. 

£ 

w  o 

CO 


CO 

as 

•4_> 

as 

u 

T3 

C 


to 
4) 
l* 

o 

as  • 
to  OS 

n  Kr  c 
— ■  bCiii 

rn  C 

<L>  £ 

2  a>  as 

3  ^u>ri 
ri,  oO  as 

00  u  dj 

as  x 
J  C5 
'M  as 

>«  E  c 

Oj  o 

as  S 
c  >- 


03 

OS 

■4—1 

u 

as 

> 


V 

to 

as 


v 


H 


to 

4) 

i- 

o 

S 

o 


o  5 

to 

n  0 
u 

as 

£>  u 

as  *0 


4) 

to  .5 

OS 

04  8  8 

Jg 


^  U  £:=: 


© 

Vh 

be 


as 

c 


as 

OS 

-4— » 

as 

> 

as 

4C 

H 


as 

as 

as 

u 


* 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


53 


o 


■ft 

to 


§ 

to 


o 

o 

CO 


CD 

be 

cd 

•*-> 

G  G 

<D  O 

a 

Ui 

C-H  -4-J 

03 

(D  u 


_  r~; 

•  t—t 

4-> 

~a 

c/D 

G 

£ 

C/3 

o 

CD 

r| 

U 

"lO 

O 

gg 

cd 

c/3 

OJ 

u 

3 

be 


<u 

G 


& 

cd  <D 
bO*4  £ 


("  S  HI 


o 

£ 

CD 

be 

r-* 

to 

•  Z-3 

H 

■4— > 

P* 

O 

*s 

u 

o 

CD 

o 

T-t 

G 

c/D 

u 

O 

o 

to 


03 

a 


v 

> 

OJ 

I — < 

H 


G 

3 


s 


o 

m 

to 


o 

o 

to 


o 

•o 

ct 


CD 

to 

cd 


8  ° 
a3 

a  « 

<d  S 


CD 

£ 

o 

w 


CD 

G 


m 

^  «G 

<u 

cd  <d 

o 

Ih 

o 

3 

be4-*  ^ 

(M 

be 

-Q 

<u  be 

E 

GG  .S  <L> 
-*-*  >  DG 
_  .G  +-> 

o 

G  cd 

to 

*  CJ 

t4 

«U  £! 

G 

«  c/D 

g 

Vh 

—  o 

T-t 

rtf  •  -h 

O  C 

o 

to 


Ih 
CD 
b*  ' 

CD 

-C 

H 


CD 

to 


54 


MARIE  HACKL  MEANS 


90 


80 

80 


10 


10 


0 


50  100  150  200  230  300  350 

Figure  6 

The  vertical  line  in  the  graph  shows  the  percentage  of  graduates 
receiving  the  scores  indicated  on  the  base  line. 

each  test.  Obviously,  if  the  possible  grades  on  the  opposites  test 
range  from  zero  to  four  hundred  and  thirty-eight  and  on  the 
hypothetical  directions  test  from  zero  to  sixteen,  to  add  or 
average  the  two  scores  made  by  any  individual  would  practically 
amount  to  weighting  the  opposites  test.  But  the  percentile 
scores  can  be  averaged  and  the  final  result  will  represent  a  com¬ 
bination  of  the  two  tests  to  which  equal  credit  is  given. 

In  order  to  make  clearer  the  meaning  of  the  percentile  table 
the  following  explanation  of  the  manner  in  which  it  was  derived 
is  offered.  For  instance  in  the  case  of  the  freshmen,  as  indicated 
in  Table  XVI,  all  scores  were  arranged  in  numerical  order,  be¬ 
ginning  with  the  lowest  and  ending  with  the  highest.  The  num¬ 
ber  of  scores  was  ascertained,  which  in  this  case  was  four  hun¬ 
dred  and  twenty.  Ten  percent  of  the  number  of  scores  was 
counted  off,  beginning  with  the  lowest  score.  The  score  upon 
which  the  forty  second  count  fell  became  the  tenth  percentile.  A 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


55 


glance  at  the  table  will  show  that  in  this  instance  it  was  fifty- 
eight  and  a  half.  This  means  that  this  grade  is  not  exceeded 
by  the  lowest  ten  percent  of  the  freshmen,  and  that  any  student 
making  this  grade  is  equalled  or  surpassed  by  ninety  percent  of 
the  college  students. 

An  additional  ten  percent  is  counted  off  to  secure  the  next 
higher  percentile,  and  so  on. 

TABLE  XVI 

Norms  for  College  Students  in  Terms  of  Percentile  Scores  in  the 

Group  Test 


Percentile 

Freshmen  Sophomores  Juniors 

Seniors 

Graduates 

All  Comb’d 

IO 

58.5 

66.5 

77.0 

84.0 

107.0 

68.5 

20 

74-5 

84.0 

97.0 

100.0 

124.0 

78.0 

30 

87.0 

99.0 

1 16.0 

116.5 

138.5 

IOI.O 

40 

97-0 

1 12.0 

130.5 

125.5 

151.0 

1 14-5 

50 

107.0 

125.0 

146.0 

137.5 

155.5 

128.5 

6o 

120.0 

139.9 

161.5 

147.0 

I9I-5 

142.0 

70 

132.5 

156.0 

181.0 

166.0 

200.0 

157.0 

8o 

147.0 

178.5 

203.0 

187.5 

247.0 

181.5 

90 

172.0 

214.0 

239-0 

231.0 

298.0 

220.5 

100 

303.5 

338.0 

3130 

35i.o 

3i3.o 

3510 

Number 

Tested 

420 

612 

364 

172 

60 

1628 

Frequently  comparisons  of  different  groups  are  stated  in  terms 
of  the  percent  of  individuals  of  the  first  group  reaching  or  ex¬ 
ceeding  the  twenty-five  or  seventy-five  percentile  of  the  second 
group.  In  the  table  below  the  twenty-five,  fifty,  and  seventy-five 
percentiles  are  indicated  for  each  class. 


TABLE  XVII 

Twenty-five,  Fifty,  and  Seventy-five  Percentile  Scores  for  Each  Class 


Percentile 

Freshmen  Sophomores 

Juniors 

Seniors 

Graduates 

All  Comb’d 

25 

82.5 

92.5 

106.0 

wd8.o 

136.0 

89.5 

50 

107.0 

125.0 

146.0 

137.5 

155-5 

128.5 

75 

139.0 

165.5 

192.0 

173-0 

213.0 

169.0 

The  sophomores  surpass  the  freshmen  and  they  are  in  turn 
outstripped  by  the  juniors.  The  gap  between  seniors  and  gradu¬ 
ates  is  wide;  but  the  former  are  little  if  at  all  superior  to  the 
juniors.  A  consideration  of  the  lowest  and  highest  score  made 
by  the  several  classes  reveals  the  fact  that  although  ten  percent  of 


56 


MARIE  HACKL  MEANS 


the  juniors  barely  reach  the  lowest  score  made  by  the  graduates, 
still  the  highest  score  attained  by  each  class  is  identical.  In 
Table  XVIII  is  shown  the  gradual  rise  of  the  lowest  score  real¬ 
ized  by  the  different  classes,  contrasted  with  the  irregularity  of 
the  highest  score. 

TABLE  XVIII 

Table  Showing  the  Highest  and  Lowest  Score  Made  by  Each  Class 

Freshmen  Sophomores  Juniors  Seniors  Graduates 


Lowest  Score  .  o  18.5  38.0  46.0  75.0 

Highest  Score  .  303-5  338-0  313.0  351.0  313.0 


Although  men  are  slightly  superior  in  the  test,  to  women 
belongs  the  highest  as  well  as  the  lowest  score.  The  greatest 
sex  difference  is  to  be  found  in  the  fortieth  and  fiftieth  percentiles 
which  in  each  case  are  surpassed  by  the  next  lowest  percentile 
secured  by  the  men.  The  table  below  shows  the  ten  percentiles 
for  both  sexes  and  the  number  tested  in  each  case.  It  also  in¬ 
cludes,  for  the  benefit  of  those  interested  in  sex  differences,  the 
more  commonly  employed  twenty-five  and  seventy-five  per¬ 
centiles. 


TABLE  XIX 

Percentile  Scores  for  Men  and  Women 


Percentiles 

Men 

Women 

0 

10 

72.0 

60.0 

20 

92-5 

85.5 

25 

99.0 

92.0 

30 

106.0 

98.5 

40 

122.5 

IOI.O 

50 

133-5 

1150 

60 

147-0 

139.0 

70 

163.5 

155.5 

75 

176.0 

167.0 

80 

189.0 

178.0 

90 

221.0 

222.0 

100 

338.0 

351-0 

Number  Tested 

535 

1093 

A  consideration  of  the  range  of  scores  made  by  both  sexes 
would  seem  to  indicate  that  women  have  a  wider  range  than 
men  in  the  trait  in  question.  Whereas  men  range  only  from 
i8-5  t0  338,  women  beginning  with  an  initial  score  of  o  reach 
351  points  as  their  highest  score. 


IX. 


Interpretation  of  the  Results 

Whether  the  test  is,  as  Simpson1  believes,  a  selective  thinking 
test,  we  cannot  from  the  very  nature  of  the  data  conclude.  How¬ 
ever,  we  feel  justified  in  assuming  that  it  is  a  test  success  in 
which  is  dependent  upon  native  ability  rather  than  number  of 
years  schooling.  The  higher  scores  attained  by  the  graduates 
confirm  rather  than  refute  the  contention,  for  although  there  is 
a  constant  process  of  weeding  out  the  mentally  inferior,  selection 
operates  particularly  at  the  conclusion  of  work  offered  for  the 
bachelor's  degree.  The  graphs  (figures  2,  3,  4,  5,  and  6)  reveal 
a  considerable  amount  of  overlapping  from  year  to  year.  Be¬ 
tween  ten  and  twenty  percent  of  the  freshmen  reach  the  median 
score  of  the  graduates  and  practically  thirty  percent,  the  median 
score  of  the  seniors. 

Not  without  significance  is  the  range  of  scores  of  the  several 
classes.  An  examination  of  Table  XVIII  discloses  the  fact  that 
the  highest  individual  score  obtained  by  either  a  junior  or  gradu¬ 
ate  is  surpassed  by  a  sophomore.  This  condition  is  not  to  be  at¬ 
tributed  to  the  simplicity  of  the  test,  which  might  admit  of  a 
number  of  nearly  perfect  records.  On  the  contrary  three  hundred 
and  fifty-one  is  the  highest  score  reached,  whereas  a  possible 
four  hundred  and  thirty-eight  could  be  attained  in  the  allotted 
time  were  an  individual  familiar  with  the  opposites.  The  lowest 
score  made  by  the  several  classes  ascends  regularly  from  fresh¬ 
men  to  graduates,  all  of  which  seems  to  indicate  that  instead 
of  the  scores  progressing  in  absolute  value  as  the  years  in  school 
increase,  the  limit  of  attainment  is  about  reached  in  the  freshmen 
year,  and  thereafter  a  dropping  off  of  the  poorer  students  raises 
the  percentage  of  high  scores  made  by  the  more  advanced 
students. 

The  form  of  the  distribution  of  the  scores,  skewed  as  it  is  to 

1  Simpson,  Benjamin  R.,  Op.  cit. 


58 


MARIE  HACKL  MEANS 


the  lower  end,  indicates  that  the  time  allotted  is  too  short.  Had 
eight  minutes  instead  of  six  been  allowed  for  the  test,  the  scores 
would  have  been  scattered  over  a  wider  area  and  the  form  of 
distribution  would  probably  have  approached  the  normal.  As  it 
stands  the  test  provides  a  large  number  of  possible  scores  which 
can  be  attained  by  the  college  student.  In  addition,  because  of 
its  difficulty,  it  offers  an  opportunity  of  testing  a  more  highly 
selected  group. 

Sex  differences  favor  the  male.  King  and  M’Crory2  found 
that  the  same  condition  obtained  with  the  hard  opposites  test 
they  used.  In  the  present  investigation  twice  as  many  women 
were  tested  as  men.  An  equal  number  might  alter  the  situation. 
This  difference  might  be  due  to  the  fact  that  the  normal  schools 
largely  attended  by  women  were  slightly  inferior  in  the  test  to  the 
colleges  and  universities.  In  addition,  proportionally  more 
women  belonged  to  the  freshman  class  and  fewer  were  enrolled 
in  the  graduate  school,  as  is  indicated  in  Table  XIV.  Until  an 
equal  number  of  unselected  cases  from  each  sex  is  tested,  the 
subject  is  debatable. 

2  King,  Irving,  and  M’Crory,  J.  L.,  “Freshmen  Tests  at  the  State  Uni¬ 
versity  of  Iowa,”  Jour.  Educ.  Psychol.,  1918,  IX. 


X. 


Suggestions  and  Directions  for  the  Use  of  the 

Opposites  Test 

The  following  suggestions  are  recommended  for  using  the 
test :  Distribute  the  test  sheets  with  the  face  downward  and 
read  aloud  the  written  directions  found  on  the  back  of  the  sheets. 
In  timing  the  class,  make  use  of  a  stop-watch.  Allowing  the 
point  value  of  the  word  for  correct  responses  and  half  that  value 
for  half  correct  responses,  grade  the  papers  according  to  the 
acceptable  responses  presented  in  Table  X.  Add  the  points  and 
record  this  sum  as  the  individual’s  score. 

If  an  entire  class  is  to  be  compared  with  the  norms  herein 
contained,  compute  the  median  score  of  the  class  and  compare 
with  the  fifty  percentile  to  ascertain  whether  the  class  is  above 
or  below  standard.  The  ratio  between  the  percentile  score  of 
the  individual  or  of  the  class  and  the  fifty  percentile  of  the  group 
may  be  taken  as  index  of  brightness. 

Frequently  it  is  necessary  to  have  more  than  one  list  of  oppo¬ 
sites.  To  meet  such  a  contingency,  the  list  has  been  divided  so 
that  both  parts  contain  an  equal  number  of  points  and  words. 
The  lists  are  given  below,  arranged  in  the  order  of  difficulty. 
The  assigned  point  value  is  at  the  left  of  the  stimulus. 

The  writer  feels  that  the  Opposites  Test  as  a  group  test  has 
sacrificed  to  time  an  important  element  in  that  it  fails  to  detect 
certain  aspects  of  the  subject’s  mind,  aspects  upon  which  per¬ 
sonal  efficiency  is  so  largely  dependent.  Most  significant  and 
enlightening 

i  Full 
I  Dim 
i  Blunt 

1  Pessimistic 

2  Public 
2  Spend 

2  Graceful 

3  Ancient 
3  Barbarous 


\  is  the  attitude  of  the  subject  toward  the  test. 


Despondent 

Doubtful 

Busy 

Abstract 

Foreign 

Simple 

Extravagant 

Dangerous 

Defective 


LIST  I 

5  Reveal 
5  Join 

7  Permanent 
7  Obnoxious 

7  Victorious 

8  Proficient 

8  Rigid 

9  Orthodoxy 


10  Analytical 

11  Loquacious 

11  Sacred 

12  Spurious 
14  Facility 
14  Pride 

17  Parsimony 
22  Suave 


6o 


MARIE  HACKL  MEANS 


LIST  II 


1  Negative 

3  Vague 

5  Diligent 

9  Permit 

1  After 

3  Fertile 

6  Impoverish 

10  Extrinsic 

1  Success 

4  Injurious 

7  Elation 

11  Dynamic 

1  Joy 

4  Advance 

.7  Sinful 

12  Heterogeneous 

2  Profit 

4  Create 

7  Conservative 

13  Disastrous 

2  Always 

4  Aristocratic 

7  Obscure 

15  Result 

2  Strength 

4  Rare 

8  Repulsion 

15  Adroit 

3  Expand 

3  Hinder 

5  Slovenly 

5  Stingy 

9  Imaginary 

26  Exoteric 

Whether  these 

attitudes,  which  Ruger1  has 

entitled  subjective 

and  objective,  so  easily  perceived  by  the  experimenter,  carry  over 
into  other  lines  of  work,  while  highly  probable,  is  a  matter  that 
yet  remains  to  be  proved. 

The  poor  record  in  the  Opposites  Test  does  not  necessarily 
indicate  the  subjective  attitude  but  an  exceptionally  good  record 
does  more  probably  indicate  the  absence  of  it.  The  student  be¬ 
comes  easily  confused  and  his  attention  is  divided  between  the 
opposite  to  be  given  and  fear  of  a  bad  showing.  The  score  goes 
low  on  account  of  frequent  interruptions  of  which  the  following 
are  typical :  “I  know  the  opposite  but  I  can’t  think  of  it  right 
now,”  “I  can’t  think  of  the  opposite  but  I  know  the  word  per¬ 
fectly,”  “This  is  a  vocabulary  test  and  my  vocabulary  is  limited.” 

This  type  of  subject  seldom  admits  that  he  is  familiar  neither 
with  the  word  nor  the  opposite.  He  prefers  in  the  individual 
test  to  waste  the  allotted  ten  seconds  attempting  to  create  the 
impression  that  he  knows  the  opposite  but  this  time  he  can  not 
quite  get  it.  Frequently  he  forgets  the  present  stimulus  in  re¬ 
gretting  the  one  just  missed. 

However  if  time  is  wasted  in  administering  the  test,  it  is 
more  than  counterbalanced  by  the  speed  with  which  he  makes 
his  exit.  He  does  not  tarry  to  ask  about  some  of  the  words  he 
has  failed  on,  nor  is  he  curious  about  the  records  of  others. 
With  a  hasty  explanation  of  his  own  inefficiency  he  hurries  from 
the  room,  leaving  the  experimenter  with  a  vague  sort  of  feeling 
that  the  latter  has  purposely  inflicted  a  mortal  injury. 

In  striking  contrast  is  the  subject  whose  interest  is  centered 
upon  the  test  itself.  He  works  rapidly  offering  neither  apologies 

1  Ruger,  H.  A.,  “The  Psychology  of  Efficiency,  A  rchiv.  of  Psychol.,  1910, 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


61 


nor  explanations.  Unashamed  he  denies  knowledge  of  some  of 
the  more  difficult  words,  but  rarely  does  he  depart  without  this 
knowledge.  He  is  eager  to  know  what  others  are  able  to  do 
under  the  same  conditions.  Interest  in  self  is  overshadowed  by 
interest  in  the  problem  in  hand. 

Peterson,2  with  his  Rational  Learning  Test,  notes  these  same 
attitudes,  and  the  writer  knows  of  no  better  test  in  which  to  ob¬ 
serve  them,  for  not  only  are  they  evident  during  the  administra¬ 
tion  of  the  test,  but  the  traits  in  question  can  be  easily  detected  in 
the  individual  records. 

Determination  of  the  extent  to  which  the  opposites  test  reveals 
native  ability,  is  not  attempted  in  this  research.  It  is  hoped  that 
someone  will  carry  on  the  work  here  begun,  and  by  a  series  of 
correlations  empirically  determine  the  scope  and  limitation  of  the 
test.  The  writer  is  of  the  opinion,  that  if  used  with  other  stan¬ 
dardized  tests,  the  Opposites  Test  will  be  found  of  value  in 
mental  diagnosis. 

2  Peterson,  Joseph,  “Experiments  in  Rational  Learning,”  Psychol.  Review > 
1918,  25,  462-463. 


XL 

Summary  and  Conclusions 

I.  The  Hard  Opposites  Test  herein  presented  enables  the 
instructor  within  a  brief  period  to  examine  a  large  number  of 
students.  Even  the  most  inexperienced  will  find  little  difficulty  in 
evaluating  the  papers,  so  statistically  simple  is  the  test. 

II.  Initial  experimentation  began  with  all  words  previously 
employed  but  eventually  the  list  was  narrowed  so  as  to  include 
only  fifty-four  of  these  stimuli.  Similarly  the  fourteen  original 
words,  for  the  most  part  of  greater  difficulty,  were  selected  from 
a  long  list  which  was  subjected  to  the  same  process  of  investiga¬ 
tion. 

III.  Subjects  were  tested  both  orally  and  individually  so  the 
experimenter  might  study  each  stimulus  from  the  standpoint  of 
both  time  and  accuracy.  Records  were  kept  of  each  individual 
response  and  reaction  time.  On  the  basis  of  these  records  the 
relative  difficulty  of  the  words  was  computed.  The  calculations 
are  based  on  at  least  one  hundred  records  for  each  word,  varying 
up  to  one  hundred  and  eighty-seven  for  others. 

IV.  The  acceptable  responses  were  chosen  by  five  judges, 
from  all  those  responses  given  by  the  subjects  experimented  upon. 
Additional  responses  which  suggested  themselves  to  each  judge 
were  recorded  and  in  turn  passed  on  by  the  others.  Effort  was 
made  to  have  the  responses  include  every  possible  opposite,  not 
merely  the  most  common  ones.  These  judgments  were  averaged 
and  a  value  of  o,  or  i  was  accorded  each  response. 

V.  The  relative  difficulty  of  the  words  was  determined  by 
multiplying  the  median  time  of  all  correct  and  half-correct  re¬ 
sponses  by  the  percentage  of  failures,  allowing  one-half  a  point 
for  half-correct  responses.  These  values  were  then  converted 
into  standard  deviation  and  steps  of  1/5  standard  deviation  were 
marked  off.  Having  selected  for  the  easiest  word,  a  word  which 
for  college  students  approached  zero  difficulty,  we  assigned  to  it 


STANDARDIZATION  OF  HARD  OPPOSITES  TEST 


63 


a  value  of  one,  to  the  next  step  a  value  of  two.  The  hardest 
word  in  the  list  received  a  value  of  twenty-six. 

VI.  For  convenient  use  the  test  was  converted  into  a  group 
test.  The  stimuli  were  arranged  in  the  ascending  order  of  dif¬ 
ficulty  and  printed  on  a  good  qualtiy  of  white  paper.  On  the 
back  of  the  sheet,  directions  make  clear  to  both  the  subject  and 
instructor  what  is  to  be  done.  Three  thousand  copies  of  the  test 
were  mailed  to  psychologists  in  normal  schools,  colleges,  and 
universities  throughout  the  country.  From  this  number  the 
records  of  1628  students  were  obtained,  on  the  basis  of  which 
norms  for  college  students  were  established.  These  standards 
are  presented  in  terms  of  percentiles  for  each  class,  including 
graduates,  and  for  each  sex. 

VII.  The  list  of  stimuli  has  been  divided  into  an  equal  num¬ 
ber  of  words  and  points,  for  the  convenience  of  any  experi¬ 
menter  who  may  desire  two  lists  rather  than  one.  As  it  stands, 
the  test  can  be  easily  incorporated  into  a  group  of  tests  and  com¬ 
parisons  made,  provided  the  other  tests  are  presented  in  terms 
of  percentiles.  The  test,  because  of  its  large  number  of  possible 
points  and  its  difficulty,  offers  an  opportunity  of  testing  ade¬ 
quately  a  less  highly  selected  group  than  that  represented  by  col¬ 
lege  students. 

VIII.  The  results  of  this  investigation  tend  to  show  that  suc¬ 
cess  in  the  test  is  dependent  upon  native  ability  rather  than  years 
of  schooling.  This  conclusion  seems  justifiable  in  view  of,  first, 
the  tremendous  amount  of  overlapping  among  the  several  classes, 
and,  second,  the  gradual  rise  of  the  lowest  scores  from  year  to 
year  while  the  highest  scores  remain  constant.  The  fact  that 
graduates  are  superior  to  seniors,  and  sophomores  to  freshmen 
is  to  a  considerable  extent  to  be  attributed  to  the  operation  of  the 
law  of  selection. 


64  MARIE  HACKL  MEANS 

APPENDIX 


Name . 

School . - . 

Sex . 

Underscore  the  class  of  which  you  are  a  member: 

Freshmen;  Sophomore;  Junior;  Senior;  Graduate. 

Directions  for  Giving  the  Test 

Before  distributing  the  test  blanks,  request  the  students  not  to  turn  over 
the  sheets  until  the  signal  is  given.  Ask  them  to  fill  in  the  blank  spaces  at 
the  top  of  the  page  and  then  make  clear  what  is  to  be  done  by  reading  aloud 
the  following  directions : 

On  the  other  side  of  this  page  are  a  number  of  words  beside  each  of 
Which  you  are  to  write  as  quickly  as  possible  the  exact  opposite.  For  instance 
if  the  word  “Black”  occurs  you  should  write  White.”  The  opposite  you 
write  must  belong  to  the  same  part  of  speech  as  the  word  in  the  list.  Phrases 
and  words  formed  by  prefixing  “Non”  are  counted  wrong.  Begin  at  the 
top  and  work  downward,  but  in  case  you  do  not  know  the  opposites,  pass 
on  to  the  next  word,  and  later,  if  you  have  time,  come  back  to  those  you 
have  omitted. 

You  will  be  given  just  six  minutes,  so  when  the  Instructor  says  “Ready” 
be  prepared  to  write,  and  when  he  says  “Go”  turn  over  the  sheet  and  begin. 
Continue  until  the  signal  is  given  to  stop. 


Full 

Busy 

Obscure 

Negative 

Abstract 

Proficient 

After 

Advance 

Rigid 

Dim 

Foreign 

Repulsion 

Blunt 

Create 

Imaginary 

Success 

Simple 

Permit 

Pessimistic 

Extravagant 

Orthodoxy 

Joy 

Aristocratic 

Analytical 

Public 

Rare 

Extrinsic 

Profit 

Dangerous 

Sacred 

Spend 

Slovenly 

Dynamic 

Always 

Defective 

Loquacious 

Graceful 

Stingy 

Heterogeneous 

Strength 

Reveal 

Spurious 

Ancient 

Diligent 

Disastrous 

Expand 

Join 

Facility 

Barbarous 

Impoverish 

Pride 

Hinder 

Permanent 

Result 

Despondent 

Elation 

Adroit 

Vague 

Sinful 

Parsimony 

Fertile 

Obnoxious 

Suave 

Doubtful 

Conservative 

Exoteric 

Injurious 

Victorious 

BIBLIOGRAPHY 


Bonser,  Frederick  G.,  “The  Reasoning  Ability  of  Children  of 
the  Fourth,  Fifth,  and  Sixth  School  Grades,”  Col.  Univ. 
Cont.  Educ.,  1906,  XXXVII. 

Bronner,  A.  F.,  “A  Comparative  Study  of  the  Intelligence  of 
Delinquent  Girls,”  Col.  Univ.  Cont.  Educ.,  1914,  LX VIII. 
Greene,  Harry  A.,  “A  Standardization  of  Certain  Opposites 
Tests,”  Jour.  Educ.  Psychol.,  1918,  IX,  559-566. 

Hering,  John  P.,  “Derivation  of  a  Scale  to  Measure  Abilities  in 
Scientific  Thinking,”  Jour.  Educ.  Psychol.,  1919,  IX,  417- 

43 1- 

King,  Irving,  and  Gold,  Hugo,  “A  Tentative  Standardization 
of  Certain  Opposites  Tests,”  Jour.  Educ.  Psychol.,  1916, 
vii,  459-482. 

King,  Irving,  and  M’Crory,  J.  L.,  “Freshman  Tests  at  the  State 
University  of  Iowa,”  Jour.  Educ.  Psychol.,  1918,  IX,  32-46. 
Kitson,  H.  D.,  “The  Scientific  Study  of  College  Students,” 
Psychol.  Monog.,  191 7,  XXIII. 

Norsworthy,  Naomi,  “The  Psychology  of  Mentally  Deficient 
Children,”  New  York,  Columbia  University,  1906. 
Peterson,  Joseph,  “Experiments  in  Rational  Learning," 
Psychol.  Review,  1918,  XXV,  462-463. 

Rosenow,  Curt,  “The  Analysis  of  Mental  Functions,”  Psychol. 
Monog.,  1917,  XXIV. 

Ruger,  H.  A.,  “The  Psychology  of  Efficiency,"  Archiv.  of 
Psychol.,  1910,  XV. 

Simpson,  B.  R.,  “Correlations  of  Mental  Abilities,”  Col.  Univ. 
Cont.  Educ.,  1912,  LIII. 

Thorndike,  E.  L.,  “Mental  and  Social  Measurements,”  New 
York,  1916. 

Weidensall,  Jean,  “The  Mentality  of  the  Criminal  Woman,” 
Baltimore,  1916. 

Woodworth,  R.  S.,  and  Wells,  F.  L.,  “Association  Tests,” 
Psychol.  Monog.,  1911,  LVII. 

Woody,  Clifford,  “Measurements  of  Some  Achievements  in 
Arithmetic,"  Col.  Univ.  Cont.  Educ.,  1916,  LXXX. 
Woolley,  Helen  Thompson,  and  Fisher,  Charlotte  Rust, 
“Mental  and  Physical  Measurements  of  Working  Children,” 
Psychol.  Monog.,  1914,  LXXVII. 


